Jul 22 02:27:53.445772 ip-10-0-130-249 systemd[1]: Starting Kubernetes Kubelet... Jul 22 02:27:53.854461 ip-10-0-130-249 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 02:27:53.854461 ip-10-0-130-249 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 02:27:53.854461 ip-10-0-130-249 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 02:27:53.854461 ip-10-0-130-249 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 02:27:53.854461 ip-10-0-130-249 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 02:27:53.855979 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.855891 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 02:27:53.860858 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860838 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:27:53.860858 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860858 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860861 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860864 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860867 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860874 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860877 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860880 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860883 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860885 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860890 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860892 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860895 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860897 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860900 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860902 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860905 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860907 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860910 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860912 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860915 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:27:53.860932 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860917 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860920 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860923 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860925 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860927 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860930 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860933 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860936 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860938 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860941 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860944 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860946 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860949 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860951 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860954 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860957 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860960 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860962 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860964 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860967 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:27:53.861380 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860969 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860972 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860974 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860976 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860979 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860981 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860984 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860986 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860989 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860991 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860994 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860996 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.860998 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861002 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861005 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861008 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861010 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861015 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861019 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861022 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:27:53.861859 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861025 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861027 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861030 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861033 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861036 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861038 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861041 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861044 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861047 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861051 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861054 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861057 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861060 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861062 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861065 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861068 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861070 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861074 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861076 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:27:53.862324 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861078 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861081 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861083 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861085 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861088 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861090 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861094 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861098 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861101 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861104 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861107 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861110 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861112 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861115 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861117 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861120 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861122 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861125 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861127 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861900 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:27:53.863008 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861915 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861918 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861921 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861924 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861927 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861929 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861932 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861935 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861937 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861940 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861942 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861945 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861947 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861950 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861952 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861954 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861957 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861959 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861961 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861964 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:27:53.863785 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861966 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861969 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861971 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861974 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861976 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861979 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861981 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861984 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861986 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861990 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861994 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.861998 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862002 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862005 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862007 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862010 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862013 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862016 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862018 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862021 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:27:53.864340 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862023 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862026 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862029 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862031 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862033 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862037 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862041 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862045 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862048 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862051 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862054 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862058 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862060 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862063 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862065 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862069 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862071 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862073 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862076 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:27:53.864845 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862079 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862081 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862084 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862086 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862089 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862092 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862101 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862104 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862107 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862109 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862112 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862115 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862118 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862120 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862123 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862125 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862128 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862130 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862132 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:27:53.865291 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862135 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862138 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862142 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862145 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862147 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862150 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862152 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862155 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862158 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862160 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862163 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862165 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862168 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862170 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862173 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862175 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862178 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862181 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862183 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:27:53.865749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.862186 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864093 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864106 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864110 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864113 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864116 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864119 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864122 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864125 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864127 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864130 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864132 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864135 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864137 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864140 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864142 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864145 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864147 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864153 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864157 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:27:53.866287 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864160 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864163 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864165 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864168 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864170 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864173 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864175 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864178 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864180 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864183 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864186 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864189 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864192 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864199 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864203 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864206 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864209 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864211 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864214 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864216 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:27:53.866775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864218 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864222 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864224 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864226 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864229 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864231 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864233 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864236 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864238 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864241 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864243 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864246 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864248 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864251 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864253 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864256 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864258 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864260 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864263 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864265 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:27:53.867231 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864267 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864269 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864272 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864274 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864277 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864280 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864282 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864285 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864287 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864290 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864293 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864295 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864297 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864300 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864302 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864304 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864307 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864309 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864311 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864315 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:27:53.867711 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864317 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864320 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864322 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864326 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864328 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864331 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864333 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864336 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864338 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864341 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864343 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864346 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864349 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864351 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864354 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864356 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864359 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864361 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864364 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:27:53.868203 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.864366 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864454 2579 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864463 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864470 2579 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864474 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864479 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864482 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864487 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864491 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864494 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864497 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864500 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864503 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864506 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864509 2579 flags.go:64] FLAG: --cgroup-root="" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864512 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864514 2579 flags.go:64] FLAG: --client-ca-file="" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864518 2579 flags.go:64] FLAG: --cloud-provider="external" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864521 2579 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864525 2579 flags.go:64] FLAG: --cluster-domain="" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864528 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864531 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864533 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864536 2579 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 02:27:53.868665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864540 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864543 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864546 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864549 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864552 2579 flags.go:64] FLAG: --contention-profiling="false" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864555 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864558 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864562 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864565 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864569 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864572 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864575 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864578 2579 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864581 2579 flags.go:64] FLAG: --enable-server="true" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864583 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864588 2579 flags.go:64] FLAG: --event-burst="100" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864591 2579 flags.go:64] FLAG: --event-qps="50" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864594 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864597 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864599 2579 flags.go:64] FLAG: --eviction-hard="" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864603 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864606 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864609 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864611 2579 flags.go:64] FLAG: --eviction-soft="" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864614 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 02:27:53.869284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864617 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864620 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864622 2579 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864626 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864628 2579 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864631 2579 flags.go:64] FLAG: --feature-gates="" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864634 2579 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864637 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864640 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864643 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864646 2579 flags.go:64] FLAG: --healthz-port="10248" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864648 2579 flags.go:64] FLAG: --help="false" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864651 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-130-249.ec2.internal" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864654 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864657 2579 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864659 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864663 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864666 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864668 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864671 2579 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864673 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864676 2579 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864678 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864681 2579 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 02:27:53.869891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864684 2579 flags.go:64] FLAG: --kube-reserved="" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864686 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864689 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864691 2579 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864694 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864696 2579 flags.go:64] FLAG: --lock-file="" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864699 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864701 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864704 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864710 2579 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864713 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864715 2579 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864718 2579 flags.go:64] FLAG: --logging-format="text" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864721 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864724 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864726 2579 flags.go:64] FLAG: --manifest-url="" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864729 2579 flags.go:64] FLAG: --manifest-url-header="" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864733 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864736 2579 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864740 2579 flags.go:64] FLAG: --max-pods="110" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864742 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864746 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864748 2579 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864751 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864754 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 02:27:53.870476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864757 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864760 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864771 2579 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864774 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864777 2579 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864780 2579 flags.go:64] FLAG: --pod-cidr="" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864782 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864788 2579 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864790 2579 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864793 2579 flags.go:64] FLAG: --pods-per-core="0" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864796 2579 flags.go:64] FLAG: --port="10250" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864799 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864801 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-080df2cee81cb12c1" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864804 2579 flags.go:64] FLAG: --qos-reserved="" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864807 2579 flags.go:64] FLAG: --read-only-port="10255" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864810 2579 flags.go:64] FLAG: --register-node="true" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864812 2579 flags.go:64] FLAG: --register-with-taints="" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864816 2579 flags.go:64] FLAG: --registry-burst="10" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864819 2579 flags.go:64] FLAG: --registry-qps="5" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864821 2579 flags.go:64] FLAG: --reserved-cpus="" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864824 2579 flags.go:64] FLAG: --reserved-memory="" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864827 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864830 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864833 2579 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864835 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 02:27:53.871063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864838 2579 flags.go:64] FLAG: --runonce="false" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864841 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864843 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864846 2579 flags.go:64] FLAG: --seccomp-default="false" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864849 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864855 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864858 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864861 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864864 2579 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864867 2579 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864869 2579 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864872 2579 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864875 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864877 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864880 2579 flags.go:64] FLAG: --system-cgroups="" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864883 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864888 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864891 2579 flags.go:64] FLAG: --tls-cert-file="" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864893 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864897 2579 flags.go:64] FLAG: --tls-min-version="" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864899 2579 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864902 2579 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864904 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864907 2579 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864909 2579 flags.go:64] FLAG: --v="2" Jul 22 02:27:53.871710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864913 2579 flags.go:64] FLAG: --version="false" Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864917 2579 flags.go:64] FLAG: --vmodule="" Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864921 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.864924 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865045 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865049 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865053 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865055 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865060 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865064 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865067 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865069 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865074 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865077 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865080 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865083 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865086 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865089 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865091 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865094 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:27:53.872326 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865097 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865099 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865102 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865104 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865107 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865109 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865111 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865114 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865116 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865119 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865121 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865123 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865126 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865133 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865136 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865138 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865141 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865143 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865146 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865148 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:27:53.872819 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865153 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865155 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865158 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865161 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865166 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865168 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865171 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865174 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865176 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865179 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865181 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865184 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865187 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865189 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865191 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865194 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865196 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865199 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865201 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:27:53.873296 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865204 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865206 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865208 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865211 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865213 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865216 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865218 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865220 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865223 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865226 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865229 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865231 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865234 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865238 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865242 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865245 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865247 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865252 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865254 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865257 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:27:53.873759 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865260 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865262 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865265 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865272 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865274 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865277 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865279 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865282 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865284 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865287 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865289 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865291 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865294 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865296 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865299 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865301 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865304 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865306 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865309 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865311 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:27:53.874234 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865314 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:27:53.874728 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865317 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:27:53.874728 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865319 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:27:53.874728 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.865321 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:27:53.874728 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.866063 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 02:27:53.874728 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.872852 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 02:27:53.874728 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.872871 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 02:27:53.874728 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.872893 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 02:27:53.874728 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.872899 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 02:27:53.874728 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.872968 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:27:53.874728 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.872974 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:27:53.874728 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.872977 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:27:53.874728 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.872980 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:27:53.874728 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.872982 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:27:53.874728 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.872985 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:27:53.874728 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.872988 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.872991 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.872993 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.872996 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.872998 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873001 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873003 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873006 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873008 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873011 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873013 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873015 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873018 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873020 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873024 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873027 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873030 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873033 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873035 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:27:53.875094 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873037 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873040 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873042 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873045 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873049 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873051 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873054 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873056 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873059 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873061 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873064 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873066 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873069 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873071 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873074 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873076 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873079 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873081 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873084 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873086 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:27:53.875570 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873089 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873092 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873094 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873096 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873099 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873101 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873103 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873106 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873108 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873111 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873113 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873117 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873121 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873124 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873127 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873130 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873133 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873136 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873139 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873141 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:27:53.876054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873144 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873146 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873149 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873152 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873156 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873159 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873162 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873164 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873167 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873169 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873172 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873174 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873177 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873179 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873182 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873184 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873187 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873189 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873191 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873194 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:27:53.876552 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873196 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:27:53.877054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873198 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:27:53.877054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873201 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:27:53.877054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873203 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:27:53.877054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873205 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:27:53.877054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873208 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:27:53.877054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873211 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:27:53.877054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873213 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:27:53.877054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873216 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:27:53.877054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873219 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:27:53.877054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873222 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:27:53.877054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873224 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:27:53.877054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873227 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:27:53.877054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873229 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:27:53.877054 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.873234 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 02:27:53.877054 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873372 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873377 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873379 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873382 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873385 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873389 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873391 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873394 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873396 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873399 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873401 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873404 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873407 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873409 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873412 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873415 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873418 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873421 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873424 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873426 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:27:53.877436 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873429 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873431 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873434 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873437 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873453 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873456 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873458 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873461 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873464 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873466 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873469 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873471 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873473 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873476 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873478 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873481 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873483 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873485 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873488 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873490 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:27:53.877916 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873493 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873495 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873498 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873500 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873502 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873505 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873507 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873510 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873512 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873515 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873517 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873519 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873522 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873524 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873526 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873529 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873531 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873534 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873536 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873539 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:27:53.878385 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873541 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873544 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873546 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873549 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873551 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873553 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873556 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873558 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873561 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873563 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873565 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873568 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873570 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873572 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873575 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873577 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873579 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873582 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873584 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873586 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:27:53.878865 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873589 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873591 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873593 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873595 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873598 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873601 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873603 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873605 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873608 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873610 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873613 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873615 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873617 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873620 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873623 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873626 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873629 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873633 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:27:53.879330 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:53.873636 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:27:53.879783 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.873640 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 02:27:53.879783 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.873788 2579 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 02:27:53.879783 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.876790 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 02:27:53.879783 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.878260 2579 server.go:1019] "Starting client certificate rotation" Jul 22 02:27:53.879783 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.878358 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 02:27:53.879783 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.878985 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 02:27:53.880606 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.880591 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 02:27:53.881804 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.881786 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 02:27:53.891208 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.891190 2579 log.go:25] "Validated CRI v1 runtime API" Jul 22 02:27:53.893213 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.893195 2579 log.go:25] "Validated CRI v1 image API" Jul 22 02:27:53.896949 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.896933 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 02:27:53.899867 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.899839 2579 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 bce56170-de80-4457-8e8e-2df3c816d427:/dev/nvme0n1p3 e885b05d-a9d1-4449-8b50-a099d997dcfb:/dev/nvme0n1p4] Jul 22 02:27:53.899941 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.899866 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 02:27:53.904903 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.904786 2579 manager.go:216] Machine: {Timestamp:2026-07-22 02:27:53.903426854 +0000 UTC m=+0.331783592 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3204968 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e036b640aa7839ddce4571d6a7253 SystemUUID:ec2e036b-640a-a783-9ddc-e4571d6a7253 BootID:394dba55-7691-4ed4-8fe3-54e0c66e4965 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f2:58:1b:f3:59 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f2:58:1b:f3:59 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:98:f5:d5:57:f3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 02:27:53.904903 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.904898 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 02:27:53.905013 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.904980 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 02:27:53.908297 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.908267 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 02:27:53.908485 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.908299 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-249.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 02:27:53.909183 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.909167 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 02:27:53.909552 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.909541 2579 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 02:27:53.909591 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.909555 2579 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 02:27:53.909591 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.909562 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 02:27:53.910189 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.910179 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 02:27:53.911850 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.911838 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 02:27:53.914654 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.914642 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 22 02:27:53.914767 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.914758 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 02:27:53.914802 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.914791 2579 kubelet.go:486] "Attempting to sync node with API server" Jul 22 02:27:53.914802 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.914800 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 02:27:53.914858 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.914812 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 02:27:53.914858 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.914821 2579 kubelet.go:398] "Adding apiserver pod source" Jul 22 02:27:53.914858 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.914830 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 02:27:53.915220 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.915209 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 02:27:53.915265 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.915228 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 02:27:53.917921 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.917891 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 02:27:53.919313 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.919299 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 02:27:53.919379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.919329 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 02:27:53.919908 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.919897 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 02:27:53.919952 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.919912 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 02:27:53.919952 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.919918 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 02:27:53.919952 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.919929 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 02:27:53.919952 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.919937 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 02:27:53.919952 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.919943 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 02:27:53.920139 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.920130 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 02:27:53.920139 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.920139 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 02:27:53.920223 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.920145 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 02:27:53.920223 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.920150 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 02:27:53.920223 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.920165 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 02:27:53.920223 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.920173 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 02:27:53.920800 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.920790 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 02:27:53.920830 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.920805 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 02:27:53.924093 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.924080 2579 server.go:1268] "Started kubelet" Jul 22 02:27:53.924196 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.924161 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 02:27:53.924256 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.924185 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 02:27:53.924304 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.924256 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 02:27:53.925017 ip-10-0-130-249 systemd[1]: Started Kubernetes Kubelet. Jul 22 02:27:53.929384 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.929345 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 02:27:53.929675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.929657 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-249.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 02:27:53.929842 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:53.929820 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-249.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 02:27:53.930229 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:53.930207 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 02:27:53.930371 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.930353 2579 server.go:310] "Adding debug handlers to kubelet server" Jul 22 02:27:53.932116 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:53.932094 2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 02:27:53.935604 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.935510 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 02:27:53.935604 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.935524 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 02:27:53.935734 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:53.935649 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-249.ec2.internal\" not found" Jul 22 02:27:53.935948 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:53.935000 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-249.ec2.internal.18c47b9eb5673ef8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-249.ec2.internal,UID:ip-10-0-130-249.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-249.ec2.internal,},FirstTimestamp:2026-07-22 02:27:53.92405068 +0000 UTC m=+0.352407418,LastTimestamp:2026-07-22 02:27:53.92405068 +0000 UTC m=+0.352407418,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-249.ec2.internal,}" Jul 22 02:27:53.936374 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.936257 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 02:27:53.936568 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.936553 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 02:27:53.936622 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.936571 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 02:27:53.936860 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.936844 2579 reconstruct.go:97] "Volume reconstruction finished" Jul 22 02:27:53.936860 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.936860 2579 reconciler.go:29] "Reconciler: start to sync state" Jul 22 02:27:53.936967 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.936848 2579 factory.go:55] Registering systemd factory Jul 22 02:27:53.936967 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.936881 2579 factory.go:223] Registration of the systemd container factory successfully Jul 22 02:27:53.937151 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.937129 2579 factory.go:153] Registering CRI-O factory Jul 22 02:27:53.937151 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.937150 2579 factory.go:223] Registration of the crio container factory successfully Jul 22 02:27:53.937279 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.937201 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 02:27:53.937279 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.937224 2579 factory.go:103] Registering Raw factory Jul 22 02:27:53.937279 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.937242 2579 manager.go:1196] Started watching for new ooms in manager Jul 22 02:27:53.937941 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.937926 2579 manager.go:349] Starting recovery of all containers Jul 22 02:27:53.942587 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:53.942550 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 22 02:27:53.942706 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:53.942681 2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-249.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 22 02:27:53.947819 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.947644 2579 manager.go:354] Recovery completed Jul 22 02:27:53.952809 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.952797 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:27:53.955118 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.955094 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:27:53.955194 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.955129 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:27:53.955194 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.955139 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:27:53.955692 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.955677 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 02:27:53.955754 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.955692 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 02:27:53.955754 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.955714 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 22 02:27:53.955873 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.955782 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:27:53.956674 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.956660 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:27:53.956741 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.956690 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:27:53.956741 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.956705 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:27:53.957109 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.957092 2579 policy_none.go:49] "None policy: Start" Jul 22 02:27:53.957109 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.957108 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 02:27:53.957253 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.957118 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 02:27:53.957376 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:53.957304 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-249.ec2.internal.18c47b9eb7414eab default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-249.ec2.internal,UID:ip-10-0-130-249.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-249.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-249.ec2.internal,},FirstTimestamp:2026-07-22 02:27:53.955118763 +0000 UTC m=+0.383475500,LastTimestamp:2026-07-22 02:27:53.955118763 +0000 UTC m=+0.383475500,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-249.ec2.internal,}" Jul 22 02:27:53.958497 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.958481 2579 policy_none.go:47] "Start" Jul 22 02:27:53.959148 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.959133 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kjg5q" Jul 22 02:27:53.966395 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:53.966329 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-249.ec2.internal.18c47b9eb741875d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-249.ec2.internal,UID:ip-10-0-130-249.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-249.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-249.ec2.internal,},FirstTimestamp:2026-07-22 02:27:53.955133277 +0000 UTC m=+0.383490015,LastTimestamp:2026-07-22 02:27:53.955133277 +0000 UTC m=+0.383490015,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-249.ec2.internal,}" Jul 22 02:27:53.966836 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.966819 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kjg5q" Jul 22 02:27:54.006395 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.997121 2579 manager.go:337] "Starting Device Plugin manager" Jul 22 02:27:54.006395 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:53.997155 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 02:27:54.006395 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.997169 2579 server.go:87] "Starting device plugin registration server" Jul 22 02:27:54.006395 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.997405 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 02:27:54.006395 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.997416 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 02:27:54.006395 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.997496 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 02:27:54.006395 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.997571 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 02:27:54.006395 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:53.997580 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 02:27:54.006395 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:53.998580 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 02:27:54.006395 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:53.998625 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-249.ec2.internal\" not found" Jul 22 02:27:54.040341 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.040307 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 02:27:54.041652 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.041632 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 02:27:54.041705 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.041682 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 02:27:54.041748 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.041712 2579 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 02:27:54.041796 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:54.041753 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 02:27:54.044455 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.044412 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:27:54.098499 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.098467 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:27:54.099647 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.099628 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:27:54.099749 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.099661 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:27:54.099749 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.099671 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:27:54.099749 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.099694 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-249.ec2.internal" Jul 22 02:27:54.113160 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.113093 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-249.ec2.internal" Jul 22 02:27:54.113160 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:54.113122 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-249.ec2.internal\": node \"ip-10-0-130-249.ec2.internal\" not found" Jul 22 02:27:54.142012 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.141982 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-249.ec2.internal"] Jul 22 02:27:54.142147 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.142059 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:27:54.143077 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.143059 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:27:54.143172 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.143091 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:27:54.143172 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.143101 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:27:54.144389 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.144376 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:27:54.144542 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.144521 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal" Jul 22 02:27:54.144603 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.144551 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:27:54.145264 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.145235 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:27:54.145363 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.145268 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:27:54.145363 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.145285 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:27:54.146053 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.146039 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:27:54.146103 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.146064 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:27:54.146103 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.146073 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:27:54.146530 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.146516 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-249.ec2.internal" Jul 22 02:27:54.146595 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.146541 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:27:54.147615 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.147597 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:27:54.147692 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.147626 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:27:54.147692 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.147637 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:27:54.156836 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:54.156817 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-249.ec2.internal\" not found" Jul 22 02:27:54.169336 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:54.169310 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-249.ec2.internal\" not found" node="ip-10-0-130-249.ec2.internal" Jul 22 02:27:54.173914 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:54.173896 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-249.ec2.internal\" not found" node="ip-10-0-130-249.ec2.internal" Jul 22 02:27:54.237971 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.237935 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/046dd7cb569e08833c71642bc20f3e71-config\") pod \"kube-apiserver-proxy-ip-10-0-130-249.ec2.internal\" (UID: \"046dd7cb569e08833c71642bc20f3e71\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-249.ec2.internal" Jul 22 02:27:54.238141 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.237978 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e8c055ee8dcb7ea5f9e2ea3a288fed76-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal\" (UID: \"e8c055ee8dcb7ea5f9e2ea3a288fed76\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal" Jul 22 02:27:54.238141 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.238002 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8c055ee8dcb7ea5f9e2ea3a288fed76-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal\" (UID: \"e8c055ee8dcb7ea5f9e2ea3a288fed76\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal" Jul 22 02:27:54.257967 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:54.257936 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-249.ec2.internal\" not found" Jul 22 02:27:54.338846 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.338750 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e8c055ee8dcb7ea5f9e2ea3a288fed76-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal\" (UID: \"e8c055ee8dcb7ea5f9e2ea3a288fed76\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal" Jul 22 02:27:54.338846 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.338830 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e8c055ee8dcb7ea5f9e2ea3a288fed76-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal\" (UID: \"e8c055ee8dcb7ea5f9e2ea3a288fed76\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal" Jul 22 02:27:54.339063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.338869 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8c055ee8dcb7ea5f9e2ea3a288fed76-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal\" (UID: \"e8c055ee8dcb7ea5f9e2ea3a288fed76\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal" Jul 22 02:27:54.339063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.338890 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/046dd7cb569e08833c71642bc20f3e71-config\") pod \"kube-apiserver-proxy-ip-10-0-130-249.ec2.internal\" (UID: \"046dd7cb569e08833c71642bc20f3e71\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-249.ec2.internal" Jul 22 02:27:54.339063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.338942 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8c055ee8dcb7ea5f9e2ea3a288fed76-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal\" (UID: \"e8c055ee8dcb7ea5f9e2ea3a288fed76\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal" Jul 22 02:27:54.339063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.338971 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/046dd7cb569e08833c71642bc20f3e71-config\") pod \"kube-apiserver-proxy-ip-10-0-130-249.ec2.internal\" (UID: \"046dd7cb569e08833c71642bc20f3e71\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-249.ec2.internal" Jul 22 02:27:54.358932 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:54.358886 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-249.ec2.internal\" not found" Jul 22 02:27:54.459809 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:54.459736 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-249.ec2.internal\" not found" Jul 22 02:27:54.470895 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.470868 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal" Jul 22 02:27:54.478032 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.478015 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-249.ec2.internal" Jul 22 02:27:54.560665 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:54.560626 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-249.ec2.internal\" not found" Jul 22 02:27:54.661194 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:54.661156 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-249.ec2.internal\" not found" Jul 22 02:27:54.761712 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:54.761675 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-249.ec2.internal\" not found" Jul 22 02:27:54.862309 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:54.862255 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-249.ec2.internal\" not found" Jul 22 02:27:54.870250 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.870221 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:27:54.878063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.878040 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 02:27:54.878206 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.878186 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 02:27:54.878267 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.878230 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 02:27:54.963246 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:54.963205 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-249.ec2.internal\" not found" Jul 22 02:27:54.966964 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.966943 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:27:54.972031 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.971994 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 02:22:55 +0000 UTC" deadline="2028-03-20 03:38:23.328092902 +0000 UTC" Jul 22 02:27:54.972031 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:54.972029 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14569h10m28.356066636s" Jul 22 02:27:55.036419 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:55.036312 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal" Jul 22 02:27:55.042991 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:55.042945 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod046dd7cb569e08833c71642bc20f3e71.slice/crio-593b30ef2c9c948573ef9b802e69d2fc675030eb382eef29891e25e96b8cf4dd WatchSource:0}: Error finding container 593b30ef2c9c948573ef9b802e69d2fc675030eb382eef29891e25e96b8cf4dd: Status 404 returned error can't find the container with id 593b30ef2c9c948573ef9b802e69d2fc675030eb382eef29891e25e96b8cf4dd Jul 22 02:27:55.043599 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:55.043576 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c055ee8dcb7ea5f9e2ea3a288fed76.slice/crio-aecd734e8ba3463ce2a3ef5b66323332e1c7a9128108d08e65ee3e2d2b2de1bf WatchSource:0}: Error finding container aecd734e8ba3463ce2a3ef5b66323332e1c7a9128108d08e65ee3e2d2b2de1bf: Status 404 returned error can't find the container with id aecd734e8ba3463ce2a3ef5b66323332e1c7a9128108d08e65ee3e2d2b2de1bf Jul 22 02:27:55.049593 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:55.049579 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:27:55.051554 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:55.051537 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 02:27:55.052841 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:55.052827 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-249.ec2.internal" Jul 22 02:27:55.061467 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:55.061424 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 02:27:55.201284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:55.201258 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:27:57.495584 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.495497 2579 apiserver.go:52] "Watching apiserver" Jul 22 02:27:57.503170 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.503139 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 02:27:57.504804 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.504768 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-cp92t","openshift-ovn-kubernetes/ovnkube-node-nl7s9","kube-system/konnectivity-agent-j2l7h","kube-system/kube-apiserver-proxy-ip-10-0-130-249.ec2.internal","openshift-cluster-node-tuning-operator/tuned-gnnx9","openshift-image-registry/node-ca-fwbqh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal","openshift-multus/multus-additional-cni-plugins-wpncq","openshift-multus/multus-cs678","openshift-multus/network-metrics-daemon-48qrg","kube-system/global-pull-secret-syncer-cjr5p","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh","openshift-dns/node-resolver-vzq9c","openshift-network-diagnostics/network-check-target-9kvjj"] Jul 22 02:27:57.506389 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.506363 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cs678" Jul 22 02:27:57.508747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.508727 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.509694 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.509674 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vzq9c" Jul 22 02:27:57.511265 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.511244 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cp92t" Jul 22 02:27:57.512017 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.511993 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 02:27:57.512241 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.512226 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 02:27:57.512439 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.512423 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 02:27:57.512911 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.512641 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-k6wz2\"" Jul 22 02:27:57.512911 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.512853 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 02:27:57.513115 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.513101 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.513798 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.513781 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.514302 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.514280 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 02:27:57.514396 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.514322 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 02:27:57.514657 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.514575 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 02:27:57.514657 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.514624 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:27:57.514862 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.514843 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 02:27:57.516792 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.514973 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 02:27:57.516792 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.514982 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 02:27:57.516792 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.515047 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 02:27:57.516792 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.515050 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 02:27:57.516792 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.515186 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:27:57.516792 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.515254 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-47dqn\"" Jul 22 02:27:57.516792 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:57.515249 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:27:57.516792 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.515390 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 02:27:57.516792 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.515773 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 02:27:57.516792 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.516006 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 02:27:57.516792 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.516034 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hs8ng\"" Jul 22 02:27:57.516792 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.516357 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:27:57.516792 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.516360 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 02:27:57.516792 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.516503 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.518238 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.517672 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jw58b\"" Jul 22 02:27:57.518238 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.517744 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fbh22\"" Jul 22 02:27:57.518238 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.517831 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-828xl\"" Jul 22 02:27:57.518238 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.517979 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 02:27:57.518238 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.517982 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 02:27:57.518238 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.518042 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 02:27:57.518238 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.518196 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:27:57.518603 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:57.518270 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:27:57.519123 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.518867 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 02:27:57.519123 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.518965 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 02:27:57.519123 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.518867 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ksbpp\"" Jul 22 02:27:57.519540 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.519523 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fwbqh" Jul 22 02:27:57.521185 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.521166 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j2l7h" Jul 22 02:27:57.521937 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.521919 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 02:27:57.522061 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.521926 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zpbfz\"" Jul 22 02:27:57.522179 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.522156 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 02:27:57.522242 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.522203 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 02:27:57.522599 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.522565 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:27:57.522661 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:57.522623 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:27:57.523479 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.523448 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 02:27:57.524611 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.524594 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 02:27:57.525562 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.525540 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-d88sn\"" Jul 22 02:27:57.526817 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.526793 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-multus-cni-dir\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.526919 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.526852 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7d6dd73-d6a9-499a-acab-235e3a23d126-cni-binary-copy\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.526919 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.526883 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-hostroot\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.527036 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.526913 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-multus-conf-dir\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.527036 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.526969 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d18b98e3-c43e-44ff-8425-f19881254719-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.527036 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527014 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-sys\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.527176 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527047 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-run\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.527176 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527092 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlz6r\" (UniqueName: \"kubernetes.io/projected/3f5a2670-c8e6-444e-a39a-bee66e0b866a-kube-api-access-hlz6r\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.527176 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527128 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-etc-selinux\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.527176 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527157 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-tuned\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.527349 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527183 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f5a2670-c8e6-444e-a39a-bee66e0b866a-tmp\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.527349 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527210 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d18b98e3-c43e-44ff-8425-f19881254719-system-cni-dir\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.527349 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527238 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-run-netns\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.527349 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527268 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-slash\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.527349 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527296 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-os-release\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.527349 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527324 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7d6dd73-d6a9-499a-acab-235e3a23d126-multus-daemon-config\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.527349 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527350 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-run-ovn\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.527686 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527378 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-run-ovn-kubernetes\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.527686 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527403 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-registration-dir\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.527686 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527426 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5dh9\" (UniqueName: \"kubernetes.io/projected/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-kube-api-access-n5dh9\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.527686 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527452 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21af75d3-4d8f-49b7-b96f-db2dc2cd5275-tmp-dir\") pod \"node-resolver-vzq9c\" (UID: \"21af75d3-4d8f-49b7-b96f-db2dc2cd5275\") " pod="openshift-dns/node-resolver-vzq9c" Jul 22 02:27:57.527686 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527479 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/74111d74-c568-4a80-9399-bce889158263-iptables-alerter-script\") pod \"iptables-alerter-cp92t\" (UID: \"74111d74-c568-4a80-9399-bce889158263\") " pod="openshift-network-operator/iptables-alerter-cp92t" Jul 22 02:27:57.527686 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527527 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-run-netns\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.527686 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527603 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.527686 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527661 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-sys-fs\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.528037 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527692 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-log-socket\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.528037 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527722 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-modprobe-d\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.528037 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527748 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d18b98e3-c43e-44ff-8425-f19881254719-cnibin\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.528037 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527786 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bls4c\" (UniqueName: \"kubernetes.io/projected/74111d74-c568-4a80-9399-bce889158263-kube-api-access-bls4c\") pod \"iptables-alerter-cp92t\" (UID: \"74111d74-c568-4a80-9399-bce889158263\") " pod="openshift-network-operator/iptables-alerter-cp92t" Jul 22 02:27:57.528037 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527811 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-var-lib-cni-multus\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.528037 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527841 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d18b98e3-c43e-44ff-8425-f19881254719-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.528037 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527891 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-socket-dir\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.528037 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.527935 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-cnibin\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.528037 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528013 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lrbj\" (UniqueName: \"kubernetes.io/projected/f7d6dd73-d6a9-499a-acab-235e3a23d126-kube-api-access-8lrbj\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.528391 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528060 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-env-overrides\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.528391 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528086 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-ovn-node-metrics-cert\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.528391 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528114 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d18b98e3-c43e-44ff-8425-f19881254719-cni-binary-copy\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.528391 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528141 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d18b98e3-c43e-44ff-8425-f19881254719-os-release\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.528391 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528167 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/74111d74-c568-4a80-9399-bce889158263-host-slash\") pod \"iptables-alerter-cp92t\" (UID: \"74111d74-c568-4a80-9399-bce889158263\") " pod="openshift-network-operator/iptables-alerter-cp92t" Jul 22 02:27:57.528391 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528199 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-run-openvswitch\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.528391 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528250 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-cni-netd\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.528391 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528303 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21af75d3-4d8f-49b7-b96f-db2dc2cd5275-hosts-file\") pod \"node-resolver-vzq9c\" (UID: \"21af75d3-4d8f-49b7-b96f-db2dc2cd5275\") " pod="openshift-dns/node-resolver-vzq9c" Jul 22 02:27:57.528391 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528350 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-sysctl-conf\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.528391 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528373 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-systemd\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.528822 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528417 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-system-cni-dir\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.528822 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528446 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-run-k8s-cni-cncf-io\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.528822 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528477 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-node-log\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.528822 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528505 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-cni-bin\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.528822 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528569 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.528822 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528600 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct4cp\" (UniqueName: \"kubernetes.io/projected/21af75d3-4d8f-49b7-b96f-db2dc2cd5275-kube-api-access-ct4cp\") pod \"node-resolver-vzq9c\" (UID: \"21af75d3-4d8f-49b7-b96f-db2dc2cd5275\") " pod="openshift-dns/node-resolver-vzq9c" Jul 22 02:27:57.528822 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528623 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-sysconfig\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.528822 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528650 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-sysctl-d\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.528822 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528677 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj6q5\" (UniqueName: \"kubernetes.io/projected/ddc83329-034a-42bd-a8b3-9bf395c15e9b-kube-api-access-vj6q5\") pod \"network-metrics-daemon-48qrg\" (UID: \"ddc83329-034a-42bd-a8b3-9bf395c15e9b\") " pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:27:57.528822 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528705 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-var-lib-openvswitch\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.528822 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528734 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-ovnkube-config\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.528822 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528755 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-host\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.529388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528830 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l64wj\" (UniqueName: \"kubernetes.io/projected/d18b98e3-c43e-44ff-8425-f19881254719-kube-api-access-l64wj\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.529388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528859 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-run-systemd\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.529388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528884 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-kubernetes\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.529388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528911 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-var-lib-kubelet\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.529388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528937 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-var-lib-kubelet\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.529388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.528986 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-systemd-units\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.529388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.529013 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-kubelet\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.529388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.529036 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-ovnkube-script-lib\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.529388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.529063 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-546j9\" (UniqueName: \"kubernetes.io/projected/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-kube-api-access-546j9\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.529388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.529096 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs\") pod \"network-metrics-daemon-48qrg\" (UID: \"ddc83329-034a-42bd-a8b3-9bf395c15e9b\") " pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:27:57.529388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.529123 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-multus-socket-dir-parent\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.529388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.529149 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-var-lib-cni-bin\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.529388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.529172 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-etc-openvswitch\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.529388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.529204 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-run-multus-certs\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.529388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.529321 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-etc-kubernetes\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.530107 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.529437 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-lib-modules\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.530107 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.529471 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d18b98e3-c43e-44ff-8425-f19881254719-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.530107 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.529498 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-device-dir\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.531364 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.531341 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 02:27:57.552474 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.552444 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9nx55" Jul 22 02:27:57.558814 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.558788 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9nx55" Jul 22 02:27:57.617388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.617362 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 02:27:57.625579 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.625533 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal" event={"ID":"e8c055ee8dcb7ea5f9e2ea3a288fed76","Type":"ContainerStarted","Data":"aecd734e8ba3463ce2a3ef5b66323332e1c7a9128108d08e65ee3e2d2b2de1bf"} Jul 22 02:27:57.626655 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.626632 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-249.ec2.internal" event={"ID":"046dd7cb569e08833c71642bc20f3e71","Type":"ContainerStarted","Data":"593b30ef2c9c948573ef9b802e69d2fc675030eb382eef29891e25e96b8cf4dd"} Jul 22 02:27:57.630154 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630126 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlz6r\" (UniqueName: \"kubernetes.io/projected/3f5a2670-c8e6-444e-a39a-bee66e0b866a-kube-api-access-hlz6r\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.630270 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630165 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-etc-selinux\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.630270 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630184 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-tuned\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.630270 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630205 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f5a2670-c8e6-444e-a39a-bee66e0b866a-tmp\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.630435 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630268 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d18b98e3-c43e-44ff-8425-f19881254719-system-cni-dir\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.630435 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630293 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-run-netns\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.630435 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630316 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-slash\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.630435 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630340 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbg5m\" (UniqueName: \"kubernetes.io/projected/7369bc37-a20a-44e7-9af0-2b15ac190fce-kube-api-access-jbg5m\") pod \"node-ca-fwbqh\" (UID: \"7369bc37-a20a-44e7-9af0-2b15ac190fce\") " pod="openshift-image-registry/node-ca-fwbqh" Jul 22 02:27:57.630435 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630350 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-etc-selinux\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.630435 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630361 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b76d49de-e528-431e-bb2a-710c095b1779-konnectivity-ca\") pod \"konnectivity-agent-j2l7h\" (UID: \"b76d49de-e528-431e-bb2a-710c095b1779\") " pod="kube-system/konnectivity-agent-j2l7h" Jul 22 02:27:57.630435 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630380 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-run-netns\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.630747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630507 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-os-release\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.630747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630558 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 02:27:57.630747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630568 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7d6dd73-d6a9-499a-acab-235e3a23d126-multus-daemon-config\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.630747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630569 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-slash\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.630747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630591 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-run-ovn\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.630747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630616 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-run-ovn-kubernetes\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.630747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630617 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d18b98e3-c43e-44ff-8425-f19881254719-system-cni-dir\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.630747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630591 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-os-release\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.630747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630646 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-registration-dir\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.630747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630678 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-run-ovn\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.630747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630685 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5dh9\" (UniqueName: \"kubernetes.io/projected/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-kube-api-access-n5dh9\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.630747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630711 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21af75d3-4d8f-49b7-b96f-db2dc2cd5275-tmp-dir\") pod \"node-resolver-vzq9c\" (UID: \"21af75d3-4d8f-49b7-b96f-db2dc2cd5275\") " pod="openshift-dns/node-resolver-vzq9c" Jul 22 02:27:57.630747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630718 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-run-ovn-kubernetes\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.630747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630733 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/74111d74-c568-4a80-9399-bce889158263-iptables-alerter-script\") pod \"iptables-alerter-cp92t\" (UID: \"74111d74-c568-4a80-9399-bce889158263\") " pod="openshift-network-operator/iptables-alerter-cp92t" Jul 22 02:27:57.630747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630753 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-run-netns\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.631517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630774 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.631517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630796 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-sys-fs\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.631517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630836 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-log-socket\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.631517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630861 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/72d84003-0a34-4663-be5f-09d9d7a14498-dbus\") pod \"global-pull-secret-syncer-cjr5p\" (UID: \"72d84003-0a34-4663-be5f-09d9d7a14498\") " pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:27:57.631517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630883 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-modprobe-d\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.631517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630925 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d18b98e3-c43e-44ff-8425-f19881254719-cnibin\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.631517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.630971 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-registration-dir\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.631517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631001 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bls4c\" (UniqueName: \"kubernetes.io/projected/74111d74-c568-4a80-9399-bce889158263-kube-api-access-bls4c\") pod \"iptables-alerter-cp92t\" (UID: \"74111d74-c568-4a80-9399-bce889158263\") " pod="openshift-network-operator/iptables-alerter-cp92t" Jul 22 02:27:57.631517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631021 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-log-socket\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.631517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631028 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-var-lib-cni-multus\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.631517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631059 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhvdt\" (UniqueName: \"kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt\") pod \"network-check-target-9kvjj\" (UID: \"97e803aa-bf79-4624-b82d-417735a543a4\") " pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:27:57.631517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631076 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21af75d3-4d8f-49b7-b96f-db2dc2cd5275-tmp-dir\") pod \"node-resolver-vzq9c\" (UID: \"21af75d3-4d8f-49b7-b96f-db2dc2cd5275\") " pod="openshift-dns/node-resolver-vzq9c" Jul 22 02:27:57.631517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631101 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.631517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631084 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d18b98e3-c43e-44ff-8425-f19881254719-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.631517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631149 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7d6dd73-d6a9-499a-acab-235e3a23d126-multus-daemon-config\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.631517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631176 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-socket-dir\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.631517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631192 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d18b98e3-c43e-44ff-8425-f19881254719-cnibin\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.632283 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631152 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-sys-fs\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.632283 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631062 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-run-netns\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.632283 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631264 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-cnibin\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.632283 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631300 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lrbj\" (UniqueName: \"kubernetes.io/projected/f7d6dd73-d6a9-499a-acab-235e3a23d126-kube-api-access-8lrbj\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.632283 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631318 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-modprobe-d\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.632283 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631325 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-env-overrides\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.632283 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631347 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-cnibin\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.632283 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631356 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-ovn-node-metrics-cert\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.632283 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631394 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/72d84003-0a34-4663-be5f-09d9d7a14498-kubelet-config\") pod \"global-pull-secret-syncer-cjr5p\" (UID: \"72d84003-0a34-4663-be5f-09d9d7a14498\") " pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:27:57.632283 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631416 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-var-lib-cni-multus\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.632283 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631422 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d18b98e3-c43e-44ff-8425-f19881254719-cni-binary-copy\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.632283 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631436 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-socket-dir\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.632283 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631453 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d18b98e3-c43e-44ff-8425-f19881254719-os-release\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.632283 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631476 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/74111d74-c568-4a80-9399-bce889158263-host-slash\") pod \"iptables-alerter-cp92t\" (UID: \"74111d74-c568-4a80-9399-bce889158263\") " pod="openshift-network-operator/iptables-alerter-cp92t" Jul 22 02:27:57.632283 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631496 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-run-openvswitch\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.632283 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631517 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-cni-netd\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.632283 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631542 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret\") pod \"global-pull-secret-syncer-cjr5p\" (UID: \"72d84003-0a34-4663-be5f-09d9d7a14498\") " pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:27:57.632987 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631565 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21af75d3-4d8f-49b7-b96f-db2dc2cd5275-hosts-file\") pod \"node-resolver-vzq9c\" (UID: \"21af75d3-4d8f-49b7-b96f-db2dc2cd5275\") " pod="openshift-dns/node-resolver-vzq9c" Jul 22 02:27:57.632987 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631584 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-sysctl-conf\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.632987 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631604 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-systemd\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.632987 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631626 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-system-cni-dir\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.632987 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631665 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-run-k8s-cni-cncf-io\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.632987 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631689 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-node-log\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.632987 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631711 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-cni-bin\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.632987 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631735 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.632987 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631738 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d18b98e3-c43e-44ff-8425-f19881254719-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.632987 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631764 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct4cp\" (UniqueName: \"kubernetes.io/projected/21af75d3-4d8f-49b7-b96f-db2dc2cd5275-kube-api-access-ct4cp\") pod \"node-resolver-vzq9c\" (UID: \"21af75d3-4d8f-49b7-b96f-db2dc2cd5275\") " pod="openshift-dns/node-resolver-vzq9c" Jul 22 02:27:57.632987 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631770 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-env-overrides\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.632987 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631298 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/74111d74-c568-4a80-9399-bce889158263-iptables-alerter-script\") pod \"iptables-alerter-cp92t\" (UID: \"74111d74-c568-4a80-9399-bce889158263\") " pod="openshift-network-operator/iptables-alerter-cp92t" Jul 22 02:27:57.632987 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631828 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-sysconfig\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.632987 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631879 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d18b98e3-c43e-44ff-8425-f19881254719-cni-binary-copy\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.632987 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631979 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-run-k8s-cni-cncf-io\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.632987 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632004 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-sysctl-d\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.632987 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632033 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj6q5\" (UniqueName: \"kubernetes.io/projected/ddc83329-034a-42bd-a8b3-9bf395c15e9b-kube-api-access-vj6q5\") pod \"network-metrics-daemon-48qrg\" (UID: \"ddc83329-034a-42bd-a8b3-9bf395c15e9b\") " pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:27:57.633675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632058 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-var-lib-openvswitch\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.633675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632069 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.633675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632072 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-run-openvswitch\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.633675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632082 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-ovnkube-config\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.633675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632120 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-sysctl-conf\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.633675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632159 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-node-log\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.633675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632221 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-sysctl-d\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.633675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632274 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-var-lib-openvswitch\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.633675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632034 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-systemd\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.633675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632035 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-cni-bin\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.633675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.631997 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d18b98e3-c43e-44ff-8425-f19881254719-os-release\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.633675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632348 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7369bc37-a20a-44e7-9af0-2b15ac190fce-serviceca\") pod \"node-ca-fwbqh\" (UID: \"7369bc37-a20a-44e7-9af0-2b15ac190fce\") " pod="openshift-image-registry/node-ca-fwbqh" Jul 22 02:27:57.633675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632381 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-host\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.633675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632407 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l64wj\" (UniqueName: \"kubernetes.io/projected/d18b98e3-c43e-44ff-8425-f19881254719-kube-api-access-l64wj\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.633675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632443 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-cni-netd\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.633675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632485 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-run-systemd\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.633675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632510 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-ovnkube-config\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632446 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-run-systemd\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632562 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-sysconfig\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632571 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21af75d3-4d8f-49b7-b96f-db2dc2cd5275-hosts-file\") pod \"node-resolver-vzq9c\" (UID: \"21af75d3-4d8f-49b7-b96f-db2dc2cd5275\") " pod="openshift-dns/node-resolver-vzq9c" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632634 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-host\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632704 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-kubernetes\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632752 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/74111d74-c568-4a80-9399-bce889158263-host-slash\") pod \"iptables-alerter-cp92t\" (UID: \"74111d74-c568-4a80-9399-bce889158263\") " pod="openshift-network-operator/iptables-alerter-cp92t" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632752 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-system-cni-dir\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.632577 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-kubernetes\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633083 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-var-lib-kubelet\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633112 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-var-lib-kubelet\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633134 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-systemd-units\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633156 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7369bc37-a20a-44e7-9af0-2b15ac190fce-host\") pod \"node-ca-fwbqh\" (UID: \"7369bc37-a20a-44e7-9af0-2b15ac190fce\") " pod="openshift-image-registry/node-ca-fwbqh" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633180 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b76d49de-e528-431e-bb2a-710c095b1779-agent-certs\") pod \"konnectivity-agent-j2l7h\" (UID: \"b76d49de-e528-431e-bb2a-710c095b1779\") " pod="kube-system/konnectivity-agent-j2l7h" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633197 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-var-lib-kubelet\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633204 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-kubelet\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633227 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-ovnkube-script-lib\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633249 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-systemd-units\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.635559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633251 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-546j9\" (UniqueName: \"kubernetes.io/projected/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-kube-api-access-546j9\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633291 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs\") pod \"network-metrics-daemon-48qrg\" (UID: \"ddc83329-034a-42bd-a8b3-9bf395c15e9b\") " pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633320 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-multus-socket-dir-parent\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633347 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-var-lib-cni-bin\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633372 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-etc-openvswitch\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633412 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-run-multus-certs\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633436 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-etc-kubernetes\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633436 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-host-kubelet\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633459 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-lib-modules\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633485 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d18b98e3-c43e-44ff-8425-f19881254719-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633502 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-etc-openvswitch\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633513 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-device-dir\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633524 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-etc-kubernetes\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633545 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-multus-cni-dir\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633534 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-var-lib-cni-bin\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633552 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-multus-socket-dir-parent\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633558 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-run-multus-certs\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633569 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7d6dd73-d6a9-499a-acab-235e3a23d126-cni-binary-copy\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.636379 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633592 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-hostroot\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633616 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-multus-conf-dir\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:57.633622 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633640 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d18b98e3-c43e-44ff-8425-f19881254719-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633662 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-lib-modules\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633663 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-sys\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:57.633701 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs podName:ddc83329-034a-42bd-a8b3-9bf395c15e9b nodeName:}" failed. No retries permitted until 2026-07-22 02:27:58.133681936 +0000 UTC m=+2.982538768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs") pod "network-metrics-daemon-48qrg" (UID: "ddc83329-034a-42bd-a8b3-9bf395c15e9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633704 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-sys\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633720 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-run\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633753 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-device-dir\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633505 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-host-var-lib-kubelet\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633800 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3f5a2670-c8e6-444e-a39a-bee66e0b866a-run\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.633860 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-multus-cni-dir\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.634015 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-multus-conf-dir\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.634062 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7d6dd73-d6a9-499a-acab-235e3a23d126-hostroot\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.634131 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d18b98e3-c43e-44ff-8425-f19881254719-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.634148 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-ovnkube-script-lib\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.634196 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d18b98e3-c43e-44ff-8425-f19881254719-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.637209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.634364 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7d6dd73-d6a9-499a-acab-235e3a23d126-cni-binary-copy\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.638039 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.634372 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f5a2670-c8e6-444e-a39a-bee66e0b866a-tmp\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.638039 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.634423 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3f5a2670-c8e6-444e-a39a-bee66e0b866a-etc-tuned\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.638039 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.634526 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-ovn-node-metrics-cert\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.645195 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.645170 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bls4c\" (UniqueName: \"kubernetes.io/projected/74111d74-c568-4a80-9399-bce889158263-kube-api-access-bls4c\") pod \"iptables-alerter-cp92t\" (UID: \"74111d74-c568-4a80-9399-bce889158263\") " pod="openshift-network-operator/iptables-alerter-cp92t" Jul 22 02:27:57.647145 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.647115 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlz6r\" (UniqueName: \"kubernetes.io/projected/3f5a2670-c8e6-444e-a39a-bee66e0b866a-kube-api-access-hlz6r\") pod \"tuned-gnnx9\" (UID: \"3f5a2670-c8e6-444e-a39a-bee66e0b866a\") " pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.647374 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.647352 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5dh9\" (UniqueName: \"kubernetes.io/projected/de635c6f-4d27-4f37-bbc8-76c9c7dbab58-kube-api-access-n5dh9\") pod \"aws-ebs-csi-driver-node-9npqh\" (UID: \"de635c6f-4d27-4f37-bbc8-76c9c7dbab58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.648661 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.648634 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lrbj\" (UniqueName: \"kubernetes.io/projected/f7d6dd73-d6a9-499a-acab-235e3a23d126-kube-api-access-8lrbj\") pod \"multus-cs678\" (UID: \"f7d6dd73-d6a9-499a-acab-235e3a23d126\") " pod="openshift-multus/multus-cs678" Jul 22 02:27:57.648868 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.648845 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj6q5\" (UniqueName: \"kubernetes.io/projected/ddc83329-034a-42bd-a8b3-9bf395c15e9b-kube-api-access-vj6q5\") pod \"network-metrics-daemon-48qrg\" (UID: \"ddc83329-034a-42bd-a8b3-9bf395c15e9b\") " pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:27:57.649154 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.649136 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-546j9\" (UniqueName: \"kubernetes.io/projected/28495a7f-b408-4ce6-9c4a-555ab7c4c38d-kube-api-access-546j9\") pod \"ovnkube-node-nl7s9\" (UID: \"28495a7f-b408-4ce6-9c4a-555ab7c4c38d\") " pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.649347 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.649331 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct4cp\" (UniqueName: \"kubernetes.io/projected/21af75d3-4d8f-49b7-b96f-db2dc2cd5275-kube-api-access-ct4cp\") pod \"node-resolver-vzq9c\" (UID: \"21af75d3-4d8f-49b7-b96f-db2dc2cd5275\") " pod="openshift-dns/node-resolver-vzq9c" Jul 22 02:27:57.650077 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.650054 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l64wj\" (UniqueName: \"kubernetes.io/projected/d18b98e3-c43e-44ff-8425-f19881254719-kube-api-access-l64wj\") pod \"multus-additional-cni-plugins-wpncq\" (UID: \"d18b98e3-c43e-44ff-8425-f19881254719\") " pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.734764 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.734715 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7369bc37-a20a-44e7-9af0-2b15ac190fce-host\") pod \"node-ca-fwbqh\" (UID: \"7369bc37-a20a-44e7-9af0-2b15ac190fce\") " pod="openshift-image-registry/node-ca-fwbqh" Jul 22 02:27:57.734764 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.734771 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b76d49de-e528-431e-bb2a-710c095b1779-agent-certs\") pod \"konnectivity-agent-j2l7h\" (UID: \"b76d49de-e528-431e-bb2a-710c095b1779\") " pod="kube-system/konnectivity-agent-j2l7h" Jul 22 02:27:57.735031 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.734800 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7369bc37-a20a-44e7-9af0-2b15ac190fce-host\") pod \"node-ca-fwbqh\" (UID: \"7369bc37-a20a-44e7-9af0-2b15ac190fce\") " pod="openshift-image-registry/node-ca-fwbqh" Jul 22 02:27:57.735031 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.734825 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbg5m\" (UniqueName: \"kubernetes.io/projected/7369bc37-a20a-44e7-9af0-2b15ac190fce-kube-api-access-jbg5m\") pod \"node-ca-fwbqh\" (UID: \"7369bc37-a20a-44e7-9af0-2b15ac190fce\") " pod="openshift-image-registry/node-ca-fwbqh" Jul 22 02:27:57.735031 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.734863 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b76d49de-e528-431e-bb2a-710c095b1779-konnectivity-ca\") pod \"konnectivity-agent-j2l7h\" (UID: \"b76d49de-e528-431e-bb2a-710c095b1779\") " pod="kube-system/konnectivity-agent-j2l7h" Jul 22 02:27:57.735031 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.734893 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/72d84003-0a34-4663-be5f-09d9d7a14498-dbus\") pod \"global-pull-secret-syncer-cjr5p\" (UID: \"72d84003-0a34-4663-be5f-09d9d7a14498\") " pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:27:57.735031 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.734918 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhvdt\" (UniqueName: \"kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt\") pod \"network-check-target-9kvjj\" (UID: \"97e803aa-bf79-4624-b82d-417735a543a4\") " pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:27:57.735031 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.734961 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/72d84003-0a34-4663-be5f-09d9d7a14498-kubelet-config\") pod \"global-pull-secret-syncer-cjr5p\" (UID: \"72d84003-0a34-4663-be5f-09d9d7a14498\") " pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:27:57.735031 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.734986 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret\") pod \"global-pull-secret-syncer-cjr5p\" (UID: \"72d84003-0a34-4663-be5f-09d9d7a14498\") " pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:27:57.735031 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.735014 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7369bc37-a20a-44e7-9af0-2b15ac190fce-serviceca\") pod \"node-ca-fwbqh\" (UID: \"7369bc37-a20a-44e7-9af0-2b15ac190fce\") " pod="openshift-image-registry/node-ca-fwbqh" Jul 22 02:27:57.735459 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.735438 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7369bc37-a20a-44e7-9af0-2b15ac190fce-serviceca\") pod \"node-ca-fwbqh\" (UID: \"7369bc37-a20a-44e7-9af0-2b15ac190fce\") " pod="openshift-image-registry/node-ca-fwbqh" Jul 22 02:27:57.735536 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.735519 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/72d84003-0a34-4663-be5f-09d9d7a14498-kubelet-config\") pod \"global-pull-secret-syncer-cjr5p\" (UID: \"72d84003-0a34-4663-be5f-09d9d7a14498\") " pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:27:57.735591 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.735521 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/72d84003-0a34-4663-be5f-09d9d7a14498-dbus\") pod \"global-pull-secret-syncer-cjr5p\" (UID: \"72d84003-0a34-4663-be5f-09d9d7a14498\") " pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:27:57.735635 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:57.735605 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:27:57.735724 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:57.735711 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret podName:72d84003-0a34-4663-be5f-09d9d7a14498 nodeName:}" failed. No retries permitted until 2026-07-22 02:27:58.235697757 +0000 UTC m=+3.084554568 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret") pod "global-pull-secret-syncer-cjr5p" (UID: "72d84003-0a34-4663-be5f-09d9d7a14498") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:27:57.735826 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.735799 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b76d49de-e528-431e-bb2a-710c095b1779-konnectivity-ca\") pod \"konnectivity-agent-j2l7h\" (UID: \"b76d49de-e528-431e-bb2a-710c095b1779\") " pod="kube-system/konnectivity-agent-j2l7h" Jul 22 02:27:57.737589 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.737565 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b76d49de-e528-431e-bb2a-710c095b1779-agent-certs\") pod \"konnectivity-agent-j2l7h\" (UID: \"b76d49de-e528-431e-bb2a-710c095b1779\") " pod="kube-system/konnectivity-agent-j2l7h" Jul 22 02:27:57.745246 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:57.745223 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:27:57.745246 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:57.745248 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:27:57.745410 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:57.745261 2579 projected.go:196] Error preparing data for projected volume kube-api-access-vhvdt for pod openshift-network-diagnostics/network-check-target-9kvjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:27:57.745410 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:57.745333 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt podName:97e803aa-bf79-4624-b82d-417735a543a4 nodeName:}" failed. No retries permitted until 2026-07-22 02:27:58.245316673 +0000 UTC m=+3.094173487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vhvdt" (UniqueName: "kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt") pod "network-check-target-9kvjj" (UID: "97e803aa-bf79-4624-b82d-417735a543a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:27:57.747414 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.747359 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbg5m\" (UniqueName: \"kubernetes.io/projected/7369bc37-a20a-44e7-9af0-2b15ac190fce-kube-api-access-jbg5m\") pod \"node-ca-fwbqh\" (UID: \"7369bc37-a20a-44e7-9af0-2b15ac190fce\") " pod="openshift-image-registry/node-ca-fwbqh" Jul 22 02:27:57.820634 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.820586 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cs678" Jul 22 02:27:57.830027 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.830002 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:27:57.840044 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.840022 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vzq9c" Jul 22 02:27:57.848913 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.848865 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cp92t" Jul 22 02:27:57.856893 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.856870 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" Jul 22 02:27:57.862039 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.862021 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:27:57.865051 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.865029 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" Jul 22 02:27:57.871644 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.871622 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wpncq" Jul 22 02:27:57.881380 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.881351 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fwbqh" Jul 22 02:27:57.886297 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:57.886277 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j2l7h" Jul 22 02:27:58.137655 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:58.137620 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs\") pod \"network-metrics-daemon-48qrg\" (UID: \"ddc83329-034a-42bd-a8b3-9bf395c15e9b\") " pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:27:58.137858 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:58.137763 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:27:58.137858 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:58.137832 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs podName:ddc83329-034a-42bd-a8b3-9bf395c15e9b nodeName:}" failed. No retries permitted until 2026-07-22 02:27:59.13781483 +0000 UTC m=+3.986671640 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs") pod "network-metrics-daemon-48qrg" (UID: "ddc83329-034a-42bd-a8b3-9bf395c15e9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:27:58.238037 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:58.237993 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret\") pod \"global-pull-secret-syncer-cjr5p\" (UID: \"72d84003-0a34-4663-be5f-09d9d7a14498\") " pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:27:58.238209 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:58.238129 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:27:58.238259 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:58.238217 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret podName:72d84003-0a34-4663-be5f-09d9d7a14498 nodeName:}" failed. No retries permitted until 2026-07-22 02:27:59.238193394 +0000 UTC m=+4.087050217 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret") pod "global-pull-secret-syncer-cjr5p" (UID: "72d84003-0a34-4663-be5f-09d9d7a14498") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:27:58.338500 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:58.338465 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhvdt\" (UniqueName: \"kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt\") pod \"network-check-target-9kvjj\" (UID: \"97e803aa-bf79-4624-b82d-417735a543a4\") " pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:27:58.338701 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:58.338669 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:27:58.338701 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:58.338705 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:27:58.338844 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:58.338718 2579 projected.go:196] Error preparing data for projected volume kube-api-access-vhvdt for pod openshift-network-diagnostics/network-check-target-9kvjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:27:58.338844 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:58.338781 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt podName:97e803aa-bf79-4624-b82d-417735a543a4 nodeName:}" failed. No retries permitted until 2026-07-22 02:27:59.338766531 +0000 UTC m=+4.187623357 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vhvdt" (UniqueName: "kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt") pod "network-check-target-9kvjj" (UID: "97e803aa-bf79-4624-b82d-417735a543a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:27:58.561270 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:58.561223 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 02:22:57 +0000 UTC" deadline="2028-01-15 03:49:55.29129506 +0000 UTC" Jul 22 02:27:58.561270 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:58.561259 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13009h21m56.730039766s" Jul 22 02:27:58.629825 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:58.629788 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f5a2670_c8e6_444e_a39a_bee66e0b866a.slice/crio-9bb85bda5916f1f88696dff66656e14a97da6fc26417d4a10da34d435510ace6 WatchSource:0}: Error finding container 9bb85bda5916f1f88696dff66656e14a97da6fc26417d4a10da34d435510ace6: Status 404 returned error can't find the container with id 9bb85bda5916f1f88696dff66656e14a97da6fc26417d4a10da34d435510ace6 Jul 22 02:27:58.634387 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:58.634359 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7369bc37_a20a_44e7_9af0_2b15ac190fce.slice/crio-6caaeeaf7a74d720f8f18e5052125c85772c2a9c105817d517debdc01c0a64b6 WatchSource:0}: Error finding container 6caaeeaf7a74d720f8f18e5052125c85772c2a9c105817d517debdc01c0a64b6: Status 404 returned error can't find the container with id 6caaeeaf7a74d720f8f18e5052125c85772c2a9c105817d517debdc01c0a64b6 Jul 22 02:27:58.646400 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:58.646378 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:27:58.662327 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:58.662299 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde635c6f_4d27_4f37_bbc8_76c9c7dbab58.slice/crio-f8aa0eb7cbf461812240588ed62369db78d118e89a4117ed469843e3f6814335 WatchSource:0}: Error finding container f8aa0eb7cbf461812240588ed62369db78d118e89a4117ed469843e3f6814335: Status 404 returned error can't find the container with id f8aa0eb7cbf461812240588ed62369db78d118e89a4117ed469843e3f6814335 Jul 22 02:27:58.664213 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:58.664178 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7d6dd73_d6a9_499a_acab_235e3a23d126.slice/crio-ff170192d92aacc6eca54b4f66da061b3ce69e5c76d4bc3991cc38c7a8afcc36 WatchSource:0}: Error finding container ff170192d92aacc6eca54b4f66da061b3ce69e5c76d4bc3991cc38c7a8afcc36: Status 404 returned error can't find the container with id ff170192d92aacc6eca54b4f66da061b3ce69e5c76d4bc3991cc38c7a8afcc36 Jul 22 02:27:58.665625 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:58.665598 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28495a7f_b408_4ce6_9c4a_555ab7c4c38d.slice/crio-e37ab25e1a2be69c53bb8519d03078cf427c380ca4bf022daaebdbe9f905e39f WatchSource:0}: Error finding container e37ab25e1a2be69c53bb8519d03078cf427c380ca4bf022daaebdbe9f905e39f: Status 404 returned error can't find the container with id e37ab25e1a2be69c53bb8519d03078cf427c380ca4bf022daaebdbe9f905e39f Jul 22 02:27:58.665879 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:58.665851 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21af75d3_4d8f_49b7_b96f_db2dc2cd5275.slice/crio-d8860f0b836fdacd8db9492cb26c0e11100178bae96c0872982dd55407e91a2d WatchSource:0}: Error finding container d8860f0b836fdacd8db9492cb26c0e11100178bae96c0872982dd55407e91a2d: Status 404 returned error can't find the container with id d8860f0b836fdacd8db9492cb26c0e11100178bae96c0872982dd55407e91a2d Jul 22 02:27:58.667822 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:58.667670 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74111d74_c568_4a80_9399_bce889158263.slice/crio-4d13557e98149a0031ed02d5d07e44b1039831b2f4bd2f67fb9af584fb04e027 WatchSource:0}: Error finding container 4d13557e98149a0031ed02d5d07e44b1039831b2f4bd2f67fb9af584fb04e027: Status 404 returned error can't find the container with id 4d13557e98149a0031ed02d5d07e44b1039831b2f4bd2f67fb9af584fb04e027 Jul 22 02:27:58.668532 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:27:58.668510 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76d49de_e528_431e_bb2a_710c095b1779.slice/crio-a6023871cc3d438186a6c80a186d4e2ab270d809a3f76d27eafbf3e186c19a73 WatchSource:0}: Error finding container a6023871cc3d438186a6c80a186d4e2ab270d809a3f76d27eafbf3e186c19a73: Status 404 returned error can't find the container with id a6023871cc3d438186a6c80a186d4e2ab270d809a3f76d27eafbf3e186c19a73 Jul 22 02:27:59.144477 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.144225 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs\") pod \"network-metrics-daemon-48qrg\" (UID: \"ddc83329-034a-42bd-a8b3-9bf395c15e9b\") " pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:27:59.144659 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:59.144529 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:27:59.144659 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:59.144612 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs podName:ddc83329-034a-42bd-a8b3-9bf395c15e9b nodeName:}" failed. No retries permitted until 2026-07-22 02:28:01.144589428 +0000 UTC m=+5.993446251 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs") pod "network-metrics-daemon-48qrg" (UID: "ddc83329-034a-42bd-a8b3-9bf395c15e9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:27:59.245932 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.245895 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret\") pod \"global-pull-secret-syncer-cjr5p\" (UID: \"72d84003-0a34-4663-be5f-09d9d7a14498\") " pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:27:59.246119 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:59.246041 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:27:59.246119 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:59.246103 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret podName:72d84003-0a34-4663-be5f-09d9d7a14498 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:01.246086112 +0000 UTC m=+6.094942935 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret") pod "global-pull-secret-syncer-cjr5p" (UID: "72d84003-0a34-4663-be5f-09d9d7a14498") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:27:59.346937 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.346898 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhvdt\" (UniqueName: \"kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt\") pod \"network-check-target-9kvjj\" (UID: \"97e803aa-bf79-4624-b82d-417735a543a4\") " pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:27:59.347158 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:59.347129 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:27:59.347158 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:59.347152 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:27:59.347321 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:59.347164 2579 projected.go:196] Error preparing data for projected volume kube-api-access-vhvdt for pod openshift-network-diagnostics/network-check-target-9kvjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:27:59.347321 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:59.347223 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt podName:97e803aa-bf79-4624-b82d-417735a543a4 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:01.347204927 +0000 UTC m=+6.196061756 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vhvdt" (UniqueName: "kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt") pod "network-check-target-9kvjj" (UID: "97e803aa-bf79-4624-b82d-417735a543a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:27:59.561604 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.561451 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 02:22:57 +0000 UTC" deadline="2028-02-02 06:40:03.905842232 +0000 UTC" Jul 22 02:27:59.561604 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.561517 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13444h12m4.344330143s" Jul 22 02:27:59.624904 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.624874 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:27:59.625096 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:59.625010 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:27:59.625163 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.625149 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:27:59.625268 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:59.625243 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:27:59.625332 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.625319 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:27:59.625429 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:27:59.625404 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:27:59.637758 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.637722 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fwbqh" event={"ID":"7369bc37-a20a-44e7-9af0-2b15ac190fce","Type":"ContainerStarted","Data":"6caaeeaf7a74d720f8f18e5052125c85772c2a9c105817d517debdc01c0a64b6"} Jul 22 02:27:59.643015 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.642981 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" event={"ID":"3f5a2670-c8e6-444e-a39a-bee66e0b866a","Type":"ContainerStarted","Data":"9bb85bda5916f1f88696dff66656e14a97da6fc26417d4a10da34d435510ace6"} Jul 22 02:27:59.647565 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.646591 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cp92t" event={"ID":"74111d74-c568-4a80-9399-bce889158263","Type":"ContainerStarted","Data":"4d13557e98149a0031ed02d5d07e44b1039831b2f4bd2f67fb9af584fb04e027"} Jul 22 02:27:59.655101 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.655066 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cs678" event={"ID":"f7d6dd73-d6a9-499a-acab-235e3a23d126","Type":"ContainerStarted","Data":"ff170192d92aacc6eca54b4f66da061b3ce69e5c76d4bc3991cc38c7a8afcc36"} Jul 22 02:27:59.664761 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.664724 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j2l7h" event={"ID":"b76d49de-e528-431e-bb2a-710c095b1779","Type":"ContainerStarted","Data":"a6023871cc3d438186a6c80a186d4e2ab270d809a3f76d27eafbf3e186c19a73"} Jul 22 02:27:59.677443 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.677400 2579 generic.go:356] "Generic (PLEG): container finished" podID="e8c055ee8dcb7ea5f9e2ea3a288fed76" containerID="afb5a37313cab4cff240b63ee7cddb4ebbf5e637362996bbc4a78bf7764a97b7" exitCode=0 Jul 22 02:27:59.677601 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.677510 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal" event={"ID":"e8c055ee8dcb7ea5f9e2ea3a288fed76","Type":"ContainerDied","Data":"afb5a37313cab4cff240b63ee7cddb4ebbf5e637362996bbc4a78bf7764a97b7"} Jul 22 02:27:59.690218 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.690103 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-249.ec2.internal" event={"ID":"046dd7cb569e08833c71642bc20f3e71","Type":"ContainerStarted","Data":"3bda45783af7585ff135a09f958c240ca18bc8fc292b6a3e04bf503490d92731"} Jul 22 02:27:59.711898 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.711861 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vzq9c" event={"ID":"21af75d3-4d8f-49b7-b96f-db2dc2cd5275","Type":"ContainerStarted","Data":"d8860f0b836fdacd8db9492cb26c0e11100178bae96c0872982dd55407e91a2d"} Jul 22 02:27:59.717570 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.717501 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wpncq" event={"ID":"d18b98e3-c43e-44ff-8425-f19881254719","Type":"ContainerStarted","Data":"b97527ab59095fbf11f328a12189fa45ff133f5cdc1306de01e328332b193acd"} Jul 22 02:27:59.722779 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.722745 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" event={"ID":"28495a7f-b408-4ce6-9c4a-555ab7c4c38d","Type":"ContainerStarted","Data":"e37ab25e1a2be69c53bb8519d03078cf427c380ca4bf022daaebdbe9f905e39f"} Jul 22 02:27:59.730580 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:27:59.730545 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" event={"ID":"de635c6f-4d27-4f37-bbc8-76c9c7dbab58","Type":"ContainerStarted","Data":"f8aa0eb7cbf461812240588ed62369db78d118e89a4117ed469843e3f6814335"} Jul 22 02:28:00.757149 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:00.757070 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal" event={"ID":"e8c055ee8dcb7ea5f9e2ea3a288fed76","Type":"ContainerStarted","Data":"57ed1f052de031b73ee4878ad55065cb708aa1a463830b4b6098472793090b6f"} Jul 22 02:28:00.774655 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:00.773706 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-249.ec2.internal" podStartSLOduration=4.773681606 podStartE2EDuration="4.773681606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:27:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:27:59.736804264 +0000 UTC m=+4.585661098" watchObservedRunningTime="2026-07-22 02:28:00.773681606 +0000 UTC m=+5.622538439" Jul 22 02:28:01.163477 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:01.162715 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs\") pod \"network-metrics-daemon-48qrg\" (UID: \"ddc83329-034a-42bd-a8b3-9bf395c15e9b\") " pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:01.163477 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:01.162892 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:28:01.163477 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:01.162977 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs podName:ddc83329-034a-42bd-a8b3-9bf395c15e9b nodeName:}" failed. No retries permitted until 2026-07-22 02:28:05.162957434 +0000 UTC m=+10.011814258 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs") pod "network-metrics-daemon-48qrg" (UID: "ddc83329-034a-42bd-a8b3-9bf395c15e9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:28:01.264201 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:01.263656 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret\") pod \"global-pull-secret-syncer-cjr5p\" (UID: \"72d84003-0a34-4663-be5f-09d9d7a14498\") " pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:01.264201 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:01.263776 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:28:01.264201 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:01.263832 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret podName:72d84003-0a34-4663-be5f-09d9d7a14498 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:05.26381494 +0000 UTC m=+10.112671757 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret") pod "global-pull-secret-syncer-cjr5p" (UID: "72d84003-0a34-4663-be5f-09d9d7a14498") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:28:01.364492 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:01.364451 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhvdt\" (UniqueName: \"kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt\") pod \"network-check-target-9kvjj\" (UID: \"97e803aa-bf79-4624-b82d-417735a543a4\") " pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:01.364754 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:01.364662 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:28:01.364754 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:01.364684 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:28:01.364754 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:01.364696 2579 projected.go:196] Error preparing data for projected volume kube-api-access-vhvdt for pod openshift-network-diagnostics/network-check-target-9kvjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:28:01.364754 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:01.364754 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt podName:97e803aa-bf79-4624-b82d-417735a543a4 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:05.36473551 +0000 UTC m=+10.213592340 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vhvdt" (UniqueName: "kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt") pod "network-check-target-9kvjj" (UID: "97e803aa-bf79-4624-b82d-417735a543a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:28:01.624119 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:01.624085 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:01.624294 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:01.624217 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:28:01.624294 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:01.624227 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:01.624294 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:01.624255 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:01.624479 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:01.624301 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:28:01.624479 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:01.624370 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:28:03.623107 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:03.623076 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:03.623676 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:03.623082 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:03.623676 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:03.623188 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:28:03.623676 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:03.623285 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:28:03.623676 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:03.623397 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:03.623676 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:03.623486 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:28:05.197787 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:05.197746 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs\") pod \"network-metrics-daemon-48qrg\" (UID: \"ddc83329-034a-42bd-a8b3-9bf395c15e9b\") " pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:05.198205 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:05.198017 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:28:05.198205 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:05.198086 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs podName:ddc83329-034a-42bd-a8b3-9bf395c15e9b nodeName:}" failed. No retries permitted until 2026-07-22 02:28:13.198067919 +0000 UTC m=+18.046924745 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs") pod "network-metrics-daemon-48qrg" (UID: "ddc83329-034a-42bd-a8b3-9bf395c15e9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:28:05.298872 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:05.298827 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret\") pod \"global-pull-secret-syncer-cjr5p\" (UID: \"72d84003-0a34-4663-be5f-09d9d7a14498\") " pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:05.299053 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:05.299007 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:28:05.299125 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:05.299069 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret podName:72d84003-0a34-4663-be5f-09d9d7a14498 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:13.299052288 +0000 UTC m=+18.147909105 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret") pod "global-pull-secret-syncer-cjr5p" (UID: "72d84003-0a34-4663-be5f-09d9d7a14498") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:28:05.399410 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:05.399363 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhvdt\" (UniqueName: \"kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt\") pod \"network-check-target-9kvjj\" (UID: \"97e803aa-bf79-4624-b82d-417735a543a4\") " pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:05.399595 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:05.399569 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:28:05.399658 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:05.399600 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:28:05.399658 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:05.399614 2579 projected.go:196] Error preparing data for projected volume kube-api-access-vhvdt for pod openshift-network-diagnostics/network-check-target-9kvjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:28:05.399816 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:05.399689 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt podName:97e803aa-bf79-4624-b82d-417735a543a4 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:13.39966885 +0000 UTC m=+18.248525663 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vhvdt" (UniqueName: "kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt") pod "network-check-target-9kvjj" (UID: "97e803aa-bf79-4624-b82d-417735a543a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:28:05.622667 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:05.622628 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:05.622824 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:05.622751 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:05.622824 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:05.622769 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:05.622824 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:05.622770 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:28:05.623012 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:05.622854 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:28:05.623012 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:05.622937 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:28:07.623621 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:07.623083 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:07.623621 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:07.623219 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:07.623621 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:07.623083 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:07.623621 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:07.623215 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:28:07.623621 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:07.623335 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:28:07.623621 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:07.623395 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:28:09.621735 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:09.621704 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:09.621735 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:09.621722 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:09.622275 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:09.621703 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:09.622275 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:09.621811 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:28:09.622275 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:09.621907 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:28:09.622275 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:09.622022 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:28:11.622120 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:11.622081 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:11.622120 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:11.622126 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:11.622608 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:11.622081 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:11.622608 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:11.622196 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:28:11.622608 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:11.622273 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:28:11.622608 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:11.622383 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:28:13.260321 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:13.260283 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs\") pod \"network-metrics-daemon-48qrg\" (UID: \"ddc83329-034a-42bd-a8b3-9bf395c15e9b\") " pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:13.260762 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:13.260421 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:28:13.260762 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:13.260496 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs podName:ddc83329-034a-42bd-a8b3-9bf395c15e9b nodeName:}" failed. No retries permitted until 2026-07-22 02:28:29.260477095 +0000 UTC m=+34.109333919 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs") pod "network-metrics-daemon-48qrg" (UID: "ddc83329-034a-42bd-a8b3-9bf395c15e9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:28:13.361380 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:13.361330 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret\") pod \"global-pull-secret-syncer-cjr5p\" (UID: \"72d84003-0a34-4663-be5f-09d9d7a14498\") " pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:13.361527 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:13.361485 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:28:13.361582 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:13.361571 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret podName:72d84003-0a34-4663-be5f-09d9d7a14498 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:29.361547447 +0000 UTC m=+34.210404257 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret") pod "global-pull-secret-syncer-cjr5p" (UID: "72d84003-0a34-4663-be5f-09d9d7a14498") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:28:13.461776 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:13.461734 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhvdt\" (UniqueName: \"kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt\") pod \"network-check-target-9kvjj\" (UID: \"97e803aa-bf79-4624-b82d-417735a543a4\") " pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:13.461974 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:13.461933 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:28:13.462050 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:13.461977 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:28:13.462050 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:13.461991 2579 projected.go:196] Error preparing data for projected volume kube-api-access-vhvdt for pod openshift-network-diagnostics/network-check-target-9kvjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:28:13.462149 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:13.462059 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt podName:97e803aa-bf79-4624-b82d-417735a543a4 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:29.462039279 +0000 UTC m=+34.310896126 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vhvdt" (UniqueName: "kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt") pod "network-check-target-9kvjj" (UID: "97e803aa-bf79-4624-b82d-417735a543a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:28:13.622403 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:13.622362 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:13.622403 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:13.622396 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:13.622641 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:13.622373 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:13.622641 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:13.622505 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:28:13.622641 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:13.622581 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:28:13.622770 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:13.622662 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:28:15.623010 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:15.622972 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:15.623465 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:15.623018 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:15.623465 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:15.623078 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:28:15.623465 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:15.623163 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:15.623465 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:15.623262 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:28:15.623465 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:15.623380 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:28:17.621819 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.621516 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:17.622434 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.621526 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:17.622434 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:17.621933 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:28:17.622434 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.621537 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:17.622434 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:17.621984 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:28:17.622434 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:17.622123 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:28:17.794439 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.794408 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vzq9c" event={"ID":"21af75d3-4d8f-49b7-b96f-db2dc2cd5275","Type":"ContainerStarted","Data":"ce50ac8d9ddcba661935c8e433aaf1b510af38525fce1935a3b8c1be3772baa0"} Jul 22 02:28:17.796194 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.796170 2579 generic.go:356] "Generic (PLEG): container finished" podID="d18b98e3-c43e-44ff-8425-f19881254719" containerID="e07912fc124dbe7fb372c377a93368ce8b444c29c99887b592c9c4a0ff3a11d0" exitCode=0 Jul 22 02:28:17.796302 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.796259 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wpncq" event={"ID":"d18b98e3-c43e-44ff-8425-f19881254719","Type":"ContainerDied","Data":"e07912fc124dbe7fb372c377a93368ce8b444c29c99887b592c9c4a0ff3a11d0"} Jul 22 02:28:17.799049 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.799029 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 02:28:17.799432 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.799408 2579 generic.go:356] "Generic (PLEG): container finished" podID="28495a7f-b408-4ce6-9c4a-555ab7c4c38d" containerID="57e375cbfcaaa51bd43bc0907b5f6b92c4ca175571df70090583c4b1f2188394" exitCode=1 Jul 22 02:28:17.799535 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.799472 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" event={"ID":"28495a7f-b408-4ce6-9c4a-555ab7c4c38d","Type":"ContainerStarted","Data":"219bac1dee807edde6b66429e79a16db020a135d0c2f505fc4e217a237ab8be9"} Jul 22 02:28:17.799535 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.799531 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" event={"ID":"28495a7f-b408-4ce6-9c4a-555ab7c4c38d","Type":"ContainerStarted","Data":"a1c128927ec2039011baf4e516d039937918a589f50880f7b275754d8e7fe864"} Jul 22 02:28:17.799636 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.799570 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" event={"ID":"28495a7f-b408-4ce6-9c4a-555ab7c4c38d","Type":"ContainerStarted","Data":"9dc43f228d26d6d8c4fb231a7a49f34602b7d01cf73bad43de9d6351c688e75d"} Jul 22 02:28:17.799636 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.799604 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" event={"ID":"28495a7f-b408-4ce6-9c4a-555ab7c4c38d","Type":"ContainerDied","Data":"57e375cbfcaaa51bd43bc0907b5f6b92c4ca175571df70090583c4b1f2188394"} Jul 22 02:28:17.799636 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.799636 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" event={"ID":"28495a7f-b408-4ce6-9c4a-555ab7c4c38d","Type":"ContainerStarted","Data":"358c761024615bce11039af29f88d7c6b1d645fcf9244203a9ea0c968529a078"} Jul 22 02:28:17.801204 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.801181 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" event={"ID":"de635c6f-4d27-4f37-bbc8-76c9c7dbab58","Type":"ContainerStarted","Data":"dcb802c32a1c139d0134f17396754d471dbf83d238bb808daa90e402242a3d3a"} Jul 22 02:28:17.802958 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.802921 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fwbqh" event={"ID":"7369bc37-a20a-44e7-9af0-2b15ac190fce","Type":"ContainerStarted","Data":"e557247cd5d13df7109a0181305f9b1ac0383e6486dae6a53a01565ff8b56078"} Jul 22 02:28:17.804701 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.804680 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" event={"ID":"3f5a2670-c8e6-444e-a39a-bee66e0b866a","Type":"ContainerStarted","Data":"595408f13b402081026fe9b76c836cdbd50ca1d49340b761609c80e6b17ba218"} Jul 22 02:28:17.806646 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.806264 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cs678" event={"ID":"f7d6dd73-d6a9-499a-acab-235e3a23d126","Type":"ContainerStarted","Data":"3618202ce0ff0390a7f87a12e4ad7074256c6dba3345ba619a984f448816ed29"} Jul 22 02:28:17.808043 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.808021 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j2l7h" event={"ID":"b76d49de-e528-431e-bb2a-710c095b1779","Type":"ContainerStarted","Data":"a3cb00528eca1d7dc5779712ccae935dde7d946ac5eaf8a728130eada4245fc0"} Jul 22 02:28:17.813786 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.813751 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-249.ec2.internal" podStartSLOduration=21.813739927 podStartE2EDuration="21.813739927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:27:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:00.77461103 +0000 UTC m=+5.623467863" watchObservedRunningTime="2026-07-22 02:28:17.813739927 +0000 UTC m=+22.662596758" Jul 22 02:28:17.814111 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.814083 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vzq9c" podStartSLOduration=4.440328458 podStartE2EDuration="22.814076982s" totalImagesPullingTime="18.373748524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:17.813683846 +0000 UTC m=+22.662540671" watchObservedRunningTime="2026-07-22 02:28:17.814076982 +0000 UTC m=+22.662933814" Jul 22 02:28:17.848658 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.848394 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fwbqh" podStartSLOduration=3.487999191 podStartE2EDuration="21.848375262s" totalImagesPullingTime="18.360376071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:27:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:17.831180041 +0000 UTC m=+22.680036873" watchObservedRunningTime="2026-07-22 02:28:17.848375262 +0000 UTC m=+22.697232095" Jul 22 02:28:17.849520 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.849477 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-j2l7h" podStartSLOduration=11.584273484 podStartE2EDuration="21.849466029s" totalImagesPullingTime="10.265192545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:27:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:17.848695588 +0000 UTC m=+22.697552404" watchObservedRunningTime="2026-07-22 02:28:17.849466029 +0000 UTC m=+22.698322862" Jul 22 02:28:17.873161 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.873072 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cs678" podStartSLOduration=4.442966194 podStartE2EDuration="22.873056728s" totalImagesPullingTime="18.430090534s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:17.872309325 +0000 UTC m=+22.721166157" watchObservedRunningTime="2026-07-22 02:28:17.873056728 +0000 UTC m=+22.721913561" Jul 22 02:28:17.898710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:17.898664 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gnnx9" podStartSLOduration=4.487269488 podStartE2EDuration="22.898648861s" totalImagesPullingTime="18.411379373s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:17.898028946 +0000 UTC m=+22.746885778" watchObservedRunningTime="2026-07-22 02:28:17.898648861 +0000 UTC m=+22.747505695" Jul 22 02:28:18.590347 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:18.590318 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 02:28:18.812490 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:18.812449 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cp92t" event={"ID":"74111d74-c568-4a80-9399-bce889158263","Type":"ContainerStarted","Data":"b113a9d7abcdf4798ce44430386ece2a49301224831b3016ab354eebc98b772e"} Jul 22 02:28:18.815605 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:18.815566 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 02:28:18.816005 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:18.815979 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" event={"ID":"28495a7f-b408-4ce6-9c4a-555ab7c4c38d","Type":"ContainerStarted","Data":"92e431b26759060e5cbd0063a40e6c6a5509fff1872e888f137374a8d978e09d"} Jul 22 02:28:18.817829 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:18.817806 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" event={"ID":"de635c6f-4d27-4f37-bbc8-76c9c7dbab58","Type":"ContainerStarted","Data":"f66f0095db044af5bea1702943083c6c44fe7d823ab84f7f2e0d67d073abea1c"} Jul 22 02:28:18.951598 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:18.951560 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-j2l7h" Jul 22 02:28:18.952262 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:18.952233 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-j2l7h" Jul 22 02:28:18.968167 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:18.968118 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cp92t" podStartSLOduration=5.599226668 podStartE2EDuration="23.968098568s" totalImagesPullingTime="18.3688719s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:18.830624732 +0000 UTC m=+23.679481563" watchObservedRunningTime="2026-07-22 02:28:18.968098568 +0000 UTC m=+23.816955404" Jul 22 02:28:19.589550 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:19.589461 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T02:28:18.590340602Z","UUID":"0b64d244-23d6-4841-8139-a6869e767770","Handler":null,"Name":"","Endpoint":""} Jul 22 02:28:19.591208 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:19.591188 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 02:28:19.591336 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:19.591229 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 02:28:19.621995 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:19.621965 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:19.622178 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:19.621972 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:19.622178 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:19.622103 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:19.622178 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:19.622092 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:28:19.622338 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:19.622201 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:28:19.622338 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:19.622282 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:28:19.822494 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:19.822464 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" event={"ID":"de635c6f-4d27-4f37-bbc8-76c9c7dbab58","Type":"ContainerStarted","Data":"595206e6559b6d4f2a0354ddfb35878704700aaf5e0a630a7ca102f7416dd6c1"} Jul 22 02:28:19.822937 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:19.822666 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-j2l7h" Jul 22 02:28:19.823211 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:19.823191 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-j2l7h" Jul 22 02:28:19.869355 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:19.869241 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9npqh" podStartSLOduration=4.317002003 podStartE2EDuration="24.869217113s" totalImagesPullingTime="20.55221511s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:27:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:19.846713646 +0000 UTC m=+24.695570477" watchObservedRunningTime="2026-07-22 02:28:19.869217113 +0000 UTC m=+24.718073950" Jul 22 02:28:20.831401 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:20.831198 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 02:28:20.831857 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:20.831754 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" event={"ID":"28495a7f-b408-4ce6-9c4a-555ab7c4c38d","Type":"ContainerStarted","Data":"93fc2d7641a43ed823aa7aab8a8065e4980fd9f683f7fa0727bc7f5f6d848dd4"} Jul 22 02:28:21.621567 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:21.621531 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:21.621812 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:21.621531 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:21.621812 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:21.621682 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:28:21.621812 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:21.621533 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:21.621812 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:21.621743 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:28:21.621995 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:21.621826 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:28:22.838615 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:22.838401 2579 generic.go:356] "Generic (PLEG): container finished" podID="d18b98e3-c43e-44ff-8425-f19881254719" containerID="555bf48164fdfc089d66516f447b2f913790dc40360337b46053f456671ee430" exitCode=0 Jul 22 02:28:22.839490 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:22.838498 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wpncq" event={"ID":"d18b98e3-c43e-44ff-8425-f19881254719","Type":"ContainerDied","Data":"555bf48164fdfc089d66516f447b2f913790dc40360337b46053f456671ee430"} Jul 22 02:28:22.842041 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:22.842017 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 02:28:22.842398 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:22.842377 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" event={"ID":"28495a7f-b408-4ce6-9c4a-555ab7c4c38d","Type":"ContainerStarted","Data":"51a8203f3edf11d13d3accd575cab7910b860eaeab94ad2fb39cea1e523ee900"} Jul 22 02:28:22.842680 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:22.842662 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:28:22.842892 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:22.842879 2579 scope.go:117] "RemoveContainer" containerID="57e375cbfcaaa51bd43bc0907b5f6b92c4ca175571df70090583c4b1f2188394" Jul 22 02:28:22.859375 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:22.859350 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:28:23.322068 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:23.321133 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:28:23.622185 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:23.622156 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:23.622308 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:23.622164 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:23.622362 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:23.622300 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:28:23.622435 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:23.622414 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:28:23.622494 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:23.622480 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:23.622574 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:23.622555 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:28:23.847730 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:23.847501 2579 generic.go:356] "Generic (PLEG): container finished" podID="d18b98e3-c43e-44ff-8425-f19881254719" containerID="ef8d66c4b186cf49a7c23488dce6b2f7da1055d13dc0a7a2bbd299008a37b919" exitCode=0 Jul 22 02:28:23.847730 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:23.847601 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wpncq" event={"ID":"d18b98e3-c43e-44ff-8425-f19881254719","Type":"ContainerDied","Data":"ef8d66c4b186cf49a7c23488dce6b2f7da1055d13dc0a7a2bbd299008a37b919"} Jul 22 02:28:23.852592 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:23.852572 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 02:28:23.852937 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:23.852914 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" event={"ID":"28495a7f-b408-4ce6-9c4a-555ab7c4c38d","Type":"ContainerStarted","Data":"043c1faa2358821d3d50f74d3c936bc0e93752b610c94ebadd514d13a0ab273c"} Jul 22 02:28:23.853290 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:23.853270 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:28:23.868726 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:23.868700 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:28:23.910352 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:23.910300 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" podStartSLOduration=10.49812624 podStartE2EDuration="28.910284028s" totalImagesPullingTime="18.412157788s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:23.908736085 +0000 UTC m=+28.757592917" watchObservedRunningTime="2026-07-22 02:28:23.910284028 +0000 UTC m=+28.759140859" Jul 22 02:28:24.052994 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:24.052962 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9kvjj"] Jul 22 02:28:24.053169 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:24.053079 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:24.053218 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:24.053173 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:28:24.057260 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:24.057225 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cjr5p"] Jul 22 02:28:24.057376 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:24.057331 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:24.057469 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:24.057446 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:28:24.058044 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:24.058025 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-48qrg"] Jul 22 02:28:24.058121 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:24.058103 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:24.058221 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:24.058200 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:28:24.857256 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:24.857225 2579 generic.go:356] "Generic (PLEG): container finished" podID="d18b98e3-c43e-44ff-8425-f19881254719" containerID="f548d8db490b21bdf2b51f3f1b36958c39ce28f06bba1cf8682945bad4547af5" exitCode=0 Jul 22 02:28:24.857628 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:24.857260 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wpncq" event={"ID":"d18b98e3-c43e-44ff-8425-f19881254719","Type":"ContainerDied","Data":"f548d8db490b21bdf2b51f3f1b36958c39ce28f06bba1cf8682945bad4547af5"} Jul 22 02:28:25.622617 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:25.622586 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:25.622789 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:25.622713 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:25.622789 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:25.622739 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:28:25.622906 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:25.622785 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:28:25.622906 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:25.622824 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:25.622906 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:25.622877 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:28:27.622696 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:27.622465 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:27.623178 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:27.622487 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:27.623178 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:27.622791 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:28:27.623178 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:27.622906 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:28:27.623178 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:27.622487 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:27.623178 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:27.623021 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:28:29.278080 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:29.278044 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs\") pod \"network-metrics-daemon-48qrg\" (UID: \"ddc83329-034a-42bd-a8b3-9bf395c15e9b\") " pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:29.278473 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:29.278161 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:28:29.278473 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:29.278214 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs podName:ddc83329-034a-42bd-a8b3-9bf395c15e9b nodeName:}" failed. No retries permitted until 2026-07-22 02:29:01.278200512 +0000 UTC m=+66.127057322 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs") pod "network-metrics-daemon-48qrg" (UID: "ddc83329-034a-42bd-a8b3-9bf395c15e9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:28:29.378392 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:29.378353 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret\") pod \"global-pull-secret-syncer-cjr5p\" (UID: \"72d84003-0a34-4663-be5f-09d9d7a14498\") " pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:29.378566 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:29.378515 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:28:29.378610 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:29.378583 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret podName:72d84003-0a34-4663-be5f-09d9d7a14498 nodeName:}" failed. No retries permitted until 2026-07-22 02:29:01.378566714 +0000 UTC m=+66.227423527 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret") pod "global-pull-secret-syncer-cjr5p" (UID: "72d84003-0a34-4663-be5f-09d9d7a14498") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:28:29.479368 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:29.479337 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhvdt\" (UniqueName: \"kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt\") pod \"network-check-target-9kvjj\" (UID: \"97e803aa-bf79-4624-b82d-417735a543a4\") " pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:29.479526 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:29.479461 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:28:29.479526 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:29.479475 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:28:29.479526 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:29.479483 2579 projected.go:196] Error preparing data for projected volume kube-api-access-vhvdt for pod openshift-network-diagnostics/network-check-target-9kvjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:28:29.479638 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:29.479530 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt podName:97e803aa-bf79-4624-b82d-417735a543a4 nodeName:}" failed. No retries permitted until 2026-07-22 02:29:01.479519127 +0000 UTC m=+66.328375940 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vhvdt" (UniqueName: "kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt") pod "network-check-target-9kvjj" (UID: "97e803aa-bf79-4624-b82d-417735a543a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:28:29.621756 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:29.621717 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:29.621919 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:29.621720 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:29.621919 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:29.621830 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr5p" podUID="72d84003-0a34-4663-be5f-09d9d7a14498" Jul 22 02:28:29.621919 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:29.621722 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:29.622134 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:29.621918 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kvjj" podUID="97e803aa-bf79-4624-b82d-417735a543a4" Jul 22 02:28:29.622134 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:29.622040 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48qrg" podUID="ddc83329-034a-42bd-a8b3-9bf395c15e9b" Jul 22 02:28:29.872304 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:29.872273 2579 generic.go:356] "Generic (PLEG): container finished" podID="d18b98e3-c43e-44ff-8425-f19881254719" containerID="b3e8b27ed3fa275d2efcdda237d946bd3815212c407ad66583c9e46d14e6f311" exitCode=0 Jul 22 02:28:29.872304 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:29.872319 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wpncq" event={"ID":"d18b98e3-c43e-44ff-8425-f19881254719","Type":"ContainerDied","Data":"b3e8b27ed3fa275d2efcdda237d946bd3815212c407ad66583c9e46d14e6f311"} Jul 22 02:28:29.944506 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:29.944484 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-249.ec2.internal" event="NodeReady" Jul 22 02:28:29.944660 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:29.944639 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 02:28:29.983238 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:29.983200 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-fd9dff84f-67px6"] Jul 22 02:28:30.000782 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.000750 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl"] Jul 22 02:28:30.001440 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.001100 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.003557 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.003532 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 02:28:30.003827 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.003806 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 02:28:30.003925 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.003810 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-48hxd\"" Jul 22 02:28:30.004293 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.004273 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 02:28:30.021237 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.021213 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-tq2tv"] Jul 22 02:28:30.021383 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.021327 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" Jul 22 02:28:30.027336 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.027140 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 02:28:30.027336 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.027251 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 22 02:28:30.027336 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.027254 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dnrd4\"" Jul 22 02:28:30.027336 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.027300 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 22 02:28:30.027638 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.027427 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 02:28:30.033369 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.033344 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 02:28:30.039619 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.039596 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf"] Jul 22 02:28:30.039748 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.039728 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.043168 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.043150 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 22 02:28:30.043339 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.043150 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 22 02:28:30.043491 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.043171 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 02:28:30.043582 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.043194 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 02:28:30.044629 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.044612 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-pstq4\"" Jul 22 02:28:30.051461 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.051435 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 22 02:28:30.056784 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.055854 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5889846976-rpxs8"] Jul 22 02:28:30.056784 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.056151 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf" Jul 22 02:28:30.059063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.059045 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-hbbwc\"" Jul 22 02:28:30.059188 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.059048 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 22 02:28:30.059308 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.059263 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:28:30.059487 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.059472 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 22 02:28:30.059795 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.059696 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 22 02:28:30.074388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.074360 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv"] Jul 22 02:28:30.074559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.074489 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:30.077323 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.077295 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 22 02:28:30.077323 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.077296 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 02:28:30.077323 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.077323 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 22 02:28:30.077551 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.077327 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 22 02:28:30.077551 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.077384 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2z4h4\"" Jul 22 02:28:30.077651 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.077618 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 02:28:30.077711 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.077697 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 22 02:28:30.084512 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.084489 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/266db83d-8440-47b6-a7c7-8fb294a00554-registry-certificates\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.084636 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.084516 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-bound-sa-token\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.084636 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.084540 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/266db83d-8440-47b6-a7c7-8fb294a00554-ca-trust-extracted\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.084636 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.084564 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-85lxl\" (UID: \"9f11a91e-5064-4838-bd58-7ab3369cc834\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" Jul 22 02:28:30.084636 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.084620 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9f11a91e-5064-4838-bd58-7ab3369cc834-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-85lxl\" (UID: \"9f11a91e-5064-4838-bd58-7ab3369cc834\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" Jul 22 02:28:30.084755 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.084657 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/266db83d-8440-47b6-a7c7-8fb294a00554-image-registry-private-configuration\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.084755 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.084689 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs9zd\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-kube-api-access-fs9zd\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.084755 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.084710 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8bb4\" (UniqueName: \"kubernetes.io/projected/9f11a91e-5064-4838-bd58-7ab3369cc834-kube-api-access-b8bb4\") pod \"cluster-monitoring-operator-76cf8966f8-85lxl\" (UID: \"9f11a91e-5064-4838-bd58-7ab3369cc834\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" Jul 22 02:28:30.084838 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.084756 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.084838 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.084773 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/266db83d-8440-47b6-a7c7-8fb294a00554-trusted-ca\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.084838 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.084830 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/266db83d-8440-47b6-a7c7-8fb294a00554-installation-pull-secrets\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.094690 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.094666 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw"] Jul 22 02:28:30.094822 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.094792 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv" Jul 22 02:28:30.100222 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.100197 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 22 02:28:30.100998 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.100980 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-bsj4v\"" Jul 22 02:28:30.101146 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.100996 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:28:30.101146 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.101034 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 22 02:28:30.101146 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.101003 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 22 02:28:30.116785 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.116758 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-4fgl4"] Jul 22 02:28:30.116939 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.116920 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw" Jul 22 02:28:30.119830 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.119810 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vtm49\"" Jul 22 02:28:30.119937 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.119807 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 22 02:28:30.119937 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.119810 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:28:30.120319 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.120304 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 22 02:28:30.139122 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.139092 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qwwcl"] Jul 22 02:28:30.139295 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.139228 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:30.142139 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.142114 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:28:30.142256 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.142121 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 22 02:28:30.142256 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.142127 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 22 02:28:30.142349 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.142308 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-2c8rn\"" Jul 22 02:28:30.142546 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.142535 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 22 02:28:30.147500 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.147471 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 22 02:28:30.157476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.157453 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-fd9dff84f-67px6"] Jul 22 02:28:30.157599 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.157488 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6gfmw"] Jul 22 02:28:30.157599 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.157569 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qwwcl" Jul 22 02:28:30.162176 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.162154 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:28:30.162351 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.162184 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-k8hgl\"" Jul 22 02:28:30.162351 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.162275 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 22 02:28:30.171893 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.171861 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-4lxjv"] Jul 22 02:28:30.172041 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.171967 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6gfmw" Jul 22 02:28:30.174590 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.174569 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 02:28:30.174983 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.174970 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 02:28:30.175057 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.175044 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2lw7w\"" Jul 22 02:28:30.185566 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.185537 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.185712 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.185573 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhd8w\" (UniqueName: \"kubernetes.io/projected/858fe6a9-2e1c-48c7-8e67-12119323253e-kube-api-access-nhd8w\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.185712 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.185599 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3bf725ad-6951-48c4-b2e4-0d898eec59ed-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-924jv\" (UID: \"3bf725ad-6951-48c4-b2e4-0d898eec59ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv" Jul 22 02:28:30.185712 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.185651 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7jk9\" (UniqueName: \"kubernetes.io/projected/5beca28a-e7a9-410d-ae0f-dce53b6faeae-kube-api-access-b7jk9\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:30.185712 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.185676 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:28:30.185712 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.185683 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gmxvw\" (UID: \"893b2a27-72a0-418c-9db2-2383934be8e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw" Jul 22 02:28:30.185712 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.185689 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fd9dff84f-67px6: secret "image-registry-tls" not found Jul 22 02:28:30.185712 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.185705 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-stats-auth\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:30.185923 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.185722 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccwsw\" (UniqueName: \"kubernetes.io/projected/1239da74-1398-4bd0-9d28-98bacba6839a-kube-api-access-ccwsw\") pod \"service-ca-operator-5bcbc94c78-wxxgf\" (UID: \"1239da74-1398-4bd0-9d28-98bacba6839a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf" Jul 22 02:28:30.185923 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.185743 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls podName:266db83d-8440-47b6-a7c7-8fb294a00554 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:30.685727169 +0000 UTC m=+35.534584000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls") pod "image-registry-fd9dff84f-67px6" (UID: "266db83d-8440-47b6-a7c7-8fb294a00554") : secret "image-registry-tls" not found Jul 22 02:28:30.185923 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.185823 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/266db83d-8440-47b6-a7c7-8fb294a00554-installation-pull-secrets\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.185923 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.185847 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-bound-sa-token\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.185923 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.185870 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/858fe6a9-2e1c-48c7-8e67-12119323253e-serving-cert\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.185923 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.185906 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/266db83d-8440-47b6-a7c7-8fb294a00554-registry-certificates\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.186211 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.185928 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:30.186211 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.185970 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-85lxl\" (UID: \"9f11a91e-5064-4838-bd58-7ab3369cc834\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" Jul 22 02:28:30.186211 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186042 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858fe6a9-2e1c-48c7-8e67-12119323253e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.186211 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186066 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnxv6\" (UniqueName: \"kubernetes.io/projected/893b2a27-72a0-418c-9db2-2383934be8e3-kube-api-access-fnxv6\") pod \"cluster-samples-operator-79b7c55f87-gmxvw\" (UID: \"893b2a27-72a0-418c-9db2-2383934be8e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw" Jul 22 02:28:30.186211 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186096 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl"] Jul 22 02:28:30.186211 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186116 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/858fe6a9-2e1c-48c7-8e67-12119323253e-tmp\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.186211 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186158 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/266db83d-8440-47b6-a7c7-8fb294a00554-image-registry-private-configuration\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.186211 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186185 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf"] Jul 22 02:28:30.186211 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186187 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/858fe6a9-2e1c-48c7-8e67-12119323253e-snapshots\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.186211 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186199 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4lxjv" Jul 22 02:28:30.186211 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186211 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv"] Jul 22 02:28:30.186706 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186224 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:30.186706 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186233 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-tq2tv"] Jul 22 02:28:30.186706 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186248 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5889846976-rpxs8"] Jul 22 02:28:30.186706 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186254 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8bb4\" (UniqueName: \"kubernetes.io/projected/9f11a91e-5064-4838-bd58-7ab3369cc834-kube-api-access-b8bb4\") pod \"cluster-monitoring-operator-76cf8966f8-85lxl\" (UID: \"9f11a91e-5064-4838-bd58-7ab3369cc834\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" Jul 22 02:28:30.186706 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186261 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-4fgl4"] Jul 22 02:28:30.186706 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186296 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/266db83d-8440-47b6-a7c7-8fb294a00554-trusted-ca\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.186706 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186310 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6gfmw"] Jul 22 02:28:30.186706 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186364 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-x2pkg"] Jul 22 02:28:30.186706 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.186101 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:28:30.186706 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.186451 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls podName:9f11a91e-5064-4838-bd58-7ab3369cc834 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:30.686433797 +0000 UTC m=+35.535290611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-85lxl" (UID: "9f11a91e-5064-4838-bd58-7ab3369cc834") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:28:30.186706 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186572 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1239da74-1398-4bd0-9d28-98bacba6839a-config\") pod \"service-ca-operator-5bcbc94c78-wxxgf\" (UID: \"1239da74-1398-4bd0-9d28-98bacba6839a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf" Jul 22 02:28:30.186706 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186629 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1239da74-1398-4bd0-9d28-98bacba6839a-serving-cert\") pod \"service-ca-operator-5bcbc94c78-wxxgf\" (UID: \"1239da74-1398-4bd0-9d28-98bacba6839a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf" Jul 22 02:28:30.186706 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186693 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stpbs\" (UniqueName: \"kubernetes.io/projected/3bf725ad-6951-48c4-b2e4-0d898eec59ed-kube-api-access-stpbs\") pod \"kube-storage-version-migrator-operator-7658d9b749-924jv\" (UID: \"3bf725ad-6951-48c4-b2e4-0d898eec59ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv" Jul 22 02:28:30.187426 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186735 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9f11a91e-5064-4838-bd58-7ab3369cc834-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-85lxl\" (UID: \"9f11a91e-5064-4838-bd58-7ab3369cc834\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" Jul 22 02:28:30.187426 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186745 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/266db83d-8440-47b6-a7c7-8fb294a00554-registry-certificates\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.187426 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186782 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/266db83d-8440-47b6-a7c7-8fb294a00554-ca-trust-extracted\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.187426 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186812 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-default-certificate\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:30.187426 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186841 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3bf725ad-6951-48c4-b2e4-0d898eec59ed-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-924jv\" (UID: \"3bf725ad-6951-48c4-b2e4-0d898eec59ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv" Jul 22 02:28:30.187426 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186867 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs9zd\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-kube-api-access-fs9zd\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.187426 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.186891 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858fe6a9-2e1c-48c7-8e67-12119323253e-service-ca-bundle\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.187426 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.187382 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/266db83d-8440-47b6-a7c7-8fb294a00554-ca-trust-extracted\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.187896 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.187692 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9f11a91e-5064-4838-bd58-7ab3369cc834-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-85lxl\" (UID: \"9f11a91e-5064-4838-bd58-7ab3369cc834\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" Jul 22 02:28:30.188323 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.188303 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/266db83d-8440-47b6-a7c7-8fb294a00554-trusted-ca\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.188885 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.188865 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-bf8vm\"" Jul 22 02:28:30.189003 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.188908 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 02:28:30.189553 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.189537 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 02:28:30.191175 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.191155 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/266db83d-8440-47b6-a7c7-8fb294a00554-installation-pull-secrets\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.191260 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.191193 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/266db83d-8440-47b6-a7c7-8fb294a00554-image-registry-private-configuration\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.195684 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.195656 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-bound-sa-token\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.195814 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.195721 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8bb4\" (UniqueName: \"kubernetes.io/projected/9f11a91e-5064-4838-bd58-7ab3369cc834-kube-api-access-b8bb4\") pod \"cluster-monitoring-operator-76cf8966f8-85lxl\" (UID: \"9f11a91e-5064-4838-bd58-7ab3369cc834\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" Jul 22 02:28:30.196298 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.196275 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs9zd\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-kube-api-access-fs9zd\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.214962 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.214916 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-4lxjv"] Jul 22 02:28:30.214962 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.214966 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw"] Jul 22 02:28:30.215188 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.214997 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qwwcl"] Jul 22 02:28:30.215188 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.215011 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x2pkg"] Jul 22 02:28:30.215188 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.215069 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x2pkg" Jul 22 02:28:30.218223 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.218202 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vvdpl\"" Jul 22 02:28:30.218315 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.218242 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 02:28:30.218315 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.218244 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 02:28:30.218386 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.218202 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 02:28:30.255600 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.255570 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-66nws"] Jul 22 02:28:30.272210 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.272181 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-66nws"] Jul 22 02:28:30.272376 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.272310 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-66nws" Jul 22 02:28:30.275234 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.275204 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2s967\"" Jul 22 02:28:30.275234 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.275225 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 22 02:28:30.275234 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.275205 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 22 02:28:30.287459 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.287425 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhd8w\" (UniqueName: \"kubernetes.io/projected/858fe6a9-2e1c-48c7-8e67-12119323253e-kube-api-access-nhd8w\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.287459 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.287461 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3bf725ad-6951-48c4-b2e4-0d898eec59ed-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-924jv\" (UID: \"3bf725ad-6951-48c4-b2e4-0d898eec59ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv" Jul 22 02:28:30.288409 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.287486 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7jk9\" (UniqueName: \"kubernetes.io/projected/5beca28a-e7a9-410d-ae0f-dce53b6faeae-kube-api-access-b7jk9\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:30.288409 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.287512 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gmxvw\" (UID: \"893b2a27-72a0-418c-9db2-2383934be8e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw" Jul 22 02:28:30.288409 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.287538 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:28:30.288409 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.287562 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-stats-auth\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:30.288409 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.287585 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccwsw\" (UniqueName: \"kubernetes.io/projected/1239da74-1398-4bd0-9d28-98bacba6839a-kube-api-access-ccwsw\") pod \"service-ca-operator-5bcbc94c78-wxxgf\" (UID: \"1239da74-1398-4bd0-9d28-98bacba6839a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf" Jul 22 02:28:30.288409 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.287607 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 02:28:30.288409 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.287609 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5vwt\" (UniqueName: \"kubernetes.io/projected/2c70f851-1755-414a-afb0-f7c6ac1b38bf-kube-api-access-j5vwt\") pod \"network-check-source-7bc5b6bf85-4lxjv\" (UID: \"2c70f851-1755-414a-afb0-f7c6ac1b38bf\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4lxjv" Jul 22 02:28:30.288409 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.287674 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls podName:893b2a27-72a0-418c-9db2-2383934be8e3 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:30.787653619 +0000 UTC m=+35.636510433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gmxvw" (UID: "893b2a27-72a0-418c-9db2-2383934be8e3") : secret "samples-operator-tls" not found Jul 22 02:28:30.288409 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.287731 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/858fe6a9-2e1c-48c7-8e67-12119323253e-serving-cert\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.288409 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.287783 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:30.288409 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.287810 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7243e221-39df-4cbe-bd32-302e9744977d-tmp-dir\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:28:30.288409 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.287845 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858fe6a9-2e1c-48c7-8e67-12119323253e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.288409 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.287869 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4f98d41-0f79-4730-a09a-844c18aef0d4-config\") pod \"console-operator-565b7b569d-4fgl4\" (UID: \"d4f98d41-0f79-4730-a09a-844c18aef0d4\") " pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:30.288409 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.287891 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dltv\" (UniqueName: \"kubernetes.io/projected/7243e221-39df-4cbe-bd32-302e9744977d-kube-api-access-9dltv\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:28:30.288409 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.287913 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle podName:5beca28a-e7a9-410d-ae0f-dce53b6faeae nodeName:}" failed. No retries permitted until 2026-07-22 02:28:30.78789272 +0000 UTC m=+35.636749535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle") pod "router-default-5889846976-rpxs8" (UID: "5beca28a-e7a9-410d-ae0f-dce53b6faeae") : configmap references non-existent config key: service-ca.crt Jul 22 02:28:30.289142 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.287990 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnxv6\" (UniqueName: \"kubernetes.io/projected/893b2a27-72a0-418c-9db2-2383934be8e3-kube-api-access-fnxv6\") pod \"cluster-samples-operator-79b7c55f87-gmxvw\" (UID: \"893b2a27-72a0-418c-9db2-2383934be8e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw" Jul 22 02:28:30.289142 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.288056 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a41203b-ba6d-4c90-a657-9c4cd9a748e9-cert\") pod \"ingress-canary-x2pkg\" (UID: \"4a41203b-ba6d-4c90-a657-9c4cd9a748e9\") " pod="openshift-ingress-canary/ingress-canary-x2pkg" Jul 22 02:28:30.289142 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.288091 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/858fe6a9-2e1c-48c7-8e67-12119323253e-tmp\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.289142 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.288156 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3bf725ad-6951-48c4-b2e4-0d898eec59ed-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-924jv\" (UID: \"3bf725ad-6951-48c4-b2e4-0d898eec59ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv" Jul 22 02:28:30.289142 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.288158 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/858fe6a9-2e1c-48c7-8e67-12119323253e-snapshots\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.289142 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.288695 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/858fe6a9-2e1c-48c7-8e67-12119323253e-tmp\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.289142 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.289028 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/858fe6a9-2e1c-48c7-8e67-12119323253e-snapshots\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.289414 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.289168 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858fe6a9-2e1c-48c7-8e67-12119323253e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.289414 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.288458 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4f98d41-0f79-4730-a09a-844c18aef0d4-trusted-ca\") pod \"console-operator-565b7b569d-4fgl4\" (UID: \"d4f98d41-0f79-4730-a09a-844c18aef0d4\") " pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:30.289414 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.289298 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:30.289414 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.289341 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1239da74-1398-4bd0-9d28-98bacba6839a-config\") pod \"service-ca-operator-5bcbc94c78-wxxgf\" (UID: \"1239da74-1398-4bd0-9d28-98bacba6839a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf" Jul 22 02:28:30.289414 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.289369 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4f98d41-0f79-4730-a09a-844c18aef0d4-serving-cert\") pod \"console-operator-565b7b569d-4fgl4\" (UID: \"d4f98d41-0f79-4730-a09a-844c18aef0d4\") " pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:30.289414 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.289395 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs4wv\" (UniqueName: \"kubernetes.io/projected/4a41203b-ba6d-4c90-a657-9c4cd9a748e9-kube-api-access-fs4wv\") pod \"ingress-canary-x2pkg\" (UID: \"4a41203b-ba6d-4c90-a657-9c4cd9a748e9\") " pod="openshift-ingress-canary/ingress-canary-x2pkg" Jul 22 02:28:30.289592 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.289422 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1239da74-1398-4bd0-9d28-98bacba6839a-serving-cert\") pod \"service-ca-operator-5bcbc94c78-wxxgf\" (UID: \"1239da74-1398-4bd0-9d28-98bacba6839a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf" Jul 22 02:28:30.289592 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.289460 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h956r\" (UniqueName: \"kubernetes.io/projected/d4f98d41-0f79-4730-a09a-844c18aef0d4-kube-api-access-h956r\") pod \"console-operator-565b7b569d-4fgl4\" (UID: \"d4f98d41-0f79-4730-a09a-844c18aef0d4\") " pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:30.289592 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.289512 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7243e221-39df-4cbe-bd32-302e9744977d-config-volume\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:28:30.289592 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.289545 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stpbs\" (UniqueName: \"kubernetes.io/projected/3bf725ad-6951-48c4-b2e4-0d898eec59ed-kube-api-access-stpbs\") pod \"kube-storage-version-migrator-operator-7658d9b749-924jv\" (UID: \"3bf725ad-6951-48c4-b2e4-0d898eec59ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv" Jul 22 02:28:30.289770 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.289594 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-default-certificate\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:30.289770 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.289636 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3bf725ad-6951-48c4-b2e4-0d898eec59ed-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-924jv\" (UID: \"3bf725ad-6951-48c4-b2e4-0d898eec59ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv" Jul 22 02:28:30.289770 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.289677 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858fe6a9-2e1c-48c7-8e67-12119323253e-service-ca-bundle\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.289770 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.289710 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:28:30.289770 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.289709 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdf2m\" (UniqueName: \"kubernetes.io/projected/1d65fb9e-9e26-42f4-958d-67e85c7ddc31-kube-api-access-qdf2m\") pod \"volume-data-source-validator-78cf4c6f6-qwwcl\" (UID: \"1d65fb9e-9e26-42f4-958d-67e85c7ddc31\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qwwcl" Jul 22 02:28:30.290014 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.289783 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs podName:5beca28a-e7a9-410d-ae0f-dce53b6faeae nodeName:}" failed. No retries permitted until 2026-07-22 02:28:30.789766601 +0000 UTC m=+35.638623428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs") pod "router-default-5889846976-rpxs8" (UID: "5beca28a-e7a9-410d-ae0f-dce53b6faeae") : secret "router-metrics-certs-default" not found Jul 22 02:28:30.290349 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.290324 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1239da74-1398-4bd0-9d28-98bacba6839a-config\") pod \"service-ca-operator-5bcbc94c78-wxxgf\" (UID: \"1239da74-1398-4bd0-9d28-98bacba6839a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf" Jul 22 02:28:30.290588 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.290568 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-stats-auth\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:30.290666 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.290598 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/858fe6a9-2e1c-48c7-8e67-12119323253e-serving-cert\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.290833 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.290811 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858fe6a9-2e1c-48c7-8e67-12119323253e-service-ca-bundle\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.292587 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.292556 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-default-certificate\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:30.292587 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.292575 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1239da74-1398-4bd0-9d28-98bacba6839a-serving-cert\") pod \"service-ca-operator-5bcbc94c78-wxxgf\" (UID: \"1239da74-1398-4bd0-9d28-98bacba6839a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf" Jul 22 02:28:30.292694 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.292579 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3bf725ad-6951-48c4-b2e4-0d898eec59ed-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-924jv\" (UID: \"3bf725ad-6951-48c4-b2e4-0d898eec59ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv" Jul 22 02:28:30.299693 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.299670 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccwsw\" (UniqueName: \"kubernetes.io/projected/1239da74-1398-4bd0-9d28-98bacba6839a-kube-api-access-ccwsw\") pod \"service-ca-operator-5bcbc94c78-wxxgf\" (UID: \"1239da74-1398-4bd0-9d28-98bacba6839a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf" Jul 22 02:28:30.300047 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.300004 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7jk9\" (UniqueName: \"kubernetes.io/projected/5beca28a-e7a9-410d-ae0f-dce53b6faeae-kube-api-access-b7jk9\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:30.300047 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.300006 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhd8w\" (UniqueName: \"kubernetes.io/projected/858fe6a9-2e1c-48c7-8e67-12119323253e-kube-api-access-nhd8w\") pod \"insights-operator-555d9d7885-tq2tv\" (UID: \"858fe6a9-2e1c-48c7-8e67-12119323253e\") " pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.301527 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.301510 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stpbs\" (UniqueName: \"kubernetes.io/projected/3bf725ad-6951-48c4-b2e4-0d898eec59ed-kube-api-access-stpbs\") pod \"kube-storage-version-migrator-operator-7658d9b749-924jv\" (UID: \"3bf725ad-6951-48c4-b2e4-0d898eec59ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv" Jul 22 02:28:30.301733 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.301712 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnxv6\" (UniqueName: \"kubernetes.io/projected/893b2a27-72a0-418c-9db2-2383934be8e3-kube-api-access-fnxv6\") pod \"cluster-samples-operator-79b7c55f87-gmxvw\" (UID: \"893b2a27-72a0-418c-9db2-2383934be8e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw" Jul 22 02:28:30.390182 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.390099 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:28:30.390182 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.390142 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5vwt\" (UniqueName: \"kubernetes.io/projected/2c70f851-1755-414a-afb0-f7c6ac1b38bf-kube-api-access-j5vwt\") pod \"network-check-source-7bc5b6bf85-4lxjv\" (UID: \"2c70f851-1755-414a-afb0-f7c6ac1b38bf\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4lxjv" Jul 22 02:28:30.390182 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.390175 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7243e221-39df-4cbe-bd32-302e9744977d-tmp-dir\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:28:30.390421 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.390207 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4f98d41-0f79-4730-a09a-844c18aef0d4-config\") pod \"console-operator-565b7b569d-4fgl4\" (UID: \"d4f98d41-0f79-4730-a09a-844c18aef0d4\") " pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:30.390421 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.390230 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dltv\" (UniqueName: \"kubernetes.io/projected/7243e221-39df-4cbe-bd32-302e9744977d-kube-api-access-9dltv\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:28:30.390421 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.390253 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a41203b-ba6d-4c90-a657-9c4cd9a748e9-cert\") pod \"ingress-canary-x2pkg\" (UID: \"4a41203b-ba6d-4c90-a657-9c4cd9a748e9\") " pod="openshift-ingress-canary/ingress-canary-x2pkg" Jul 22 02:28:30.390421 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.390273 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:28:30.390421 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.390355 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls podName:7243e221-39df-4cbe-bd32-302e9744977d nodeName:}" failed. No retries permitted until 2026-07-22 02:28:30.890334177 +0000 UTC m=+35.739190989 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls") pod "dns-default-6gfmw" (UID: "7243e221-39df-4cbe-bd32-302e9744977d") : secret "dns-default-metrics-tls" not found Jul 22 02:28:30.390698 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.390280 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4f98d41-0f79-4730-a09a-844c18aef0d4-trusted-ca\") pod \"console-operator-565b7b569d-4fgl4\" (UID: \"d4f98d41-0f79-4730-a09a-844c18aef0d4\") " pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:30.390698 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.390551 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4f98d41-0f79-4730-a09a-844c18aef0d4-serving-cert\") pod \"console-operator-565b7b569d-4fgl4\" (UID: \"d4f98d41-0f79-4730-a09a-844c18aef0d4\") " pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:30.390698 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.390571 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7243e221-39df-4cbe-bd32-302e9744977d-tmp-dir\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:28:30.390698 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.390580 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs4wv\" (UniqueName: \"kubernetes.io/projected/4a41203b-ba6d-4c90-a657-9c4cd9a748e9-kube-api-access-fs4wv\") pod \"ingress-canary-x2pkg\" (UID: \"4a41203b-ba6d-4c90-a657-9c4cd9a748e9\") " pod="openshift-ingress-canary/ingress-canary-x2pkg" Jul 22 02:28:30.390698 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.390640 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h956r\" (UniqueName: \"kubernetes.io/projected/d4f98d41-0f79-4730-a09a-844c18aef0d4-kube-api-access-h956r\") pod \"console-operator-565b7b569d-4fgl4\" (UID: \"d4f98d41-0f79-4730-a09a-844c18aef0d4\") " pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:30.390698 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.390669 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7243e221-39df-4cbe-bd32-302e9744977d-config-volume\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:28:30.391034 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.390717 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-66nws\" (UID: \"f1da4aa5-c459-4bfd-9a72-5e179d8af2e3\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-66nws" Jul 22 02:28:30.391034 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.390755 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdf2m\" (UniqueName: \"kubernetes.io/projected/1d65fb9e-9e26-42f4-958d-67e85c7ddc31-kube-api-access-qdf2m\") pod \"volume-data-source-validator-78cf4c6f6-qwwcl\" (UID: \"1d65fb9e-9e26-42f4-958d-67e85c7ddc31\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qwwcl" Jul 22 02:28:30.391034 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.390808 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-66nws\" (UID: \"f1da4aa5-c459-4bfd-9a72-5e179d8af2e3\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-66nws" Jul 22 02:28:30.391178 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.391042 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4f98d41-0f79-4730-a09a-844c18aef0d4-config\") pod \"console-operator-565b7b569d-4fgl4\" (UID: \"d4f98d41-0f79-4730-a09a-844c18aef0d4\") " pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:30.391301 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.391281 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7243e221-39df-4cbe-bd32-302e9744977d-config-volume\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:28:30.391417 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.391398 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4f98d41-0f79-4730-a09a-844c18aef0d4-trusted-ca\") pod \"console-operator-565b7b569d-4fgl4\" (UID: \"d4f98d41-0f79-4730-a09a-844c18aef0d4\") " pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:30.392807 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.392784 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4f98d41-0f79-4730-a09a-844c18aef0d4-serving-cert\") pod \"console-operator-565b7b569d-4fgl4\" (UID: \"d4f98d41-0f79-4730-a09a-844c18aef0d4\") " pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:30.392916 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.392865 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-tq2tv" Jul 22 02:28:30.393002 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.392933 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a41203b-ba6d-4c90-a657-9c4cd9a748e9-cert\") pod \"ingress-canary-x2pkg\" (UID: \"4a41203b-ba6d-4c90-a657-9c4cd9a748e9\") " pod="openshift-ingress-canary/ingress-canary-x2pkg" Jul 22 02:28:30.398777 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.398749 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf" Jul 22 02:28:30.400357 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.400327 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdf2m\" (UniqueName: \"kubernetes.io/projected/1d65fb9e-9e26-42f4-958d-67e85c7ddc31-kube-api-access-qdf2m\") pod \"volume-data-source-validator-78cf4c6f6-qwwcl\" (UID: \"1d65fb9e-9e26-42f4-958d-67e85c7ddc31\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qwwcl" Jul 22 02:28:30.400357 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.400350 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dltv\" (UniqueName: \"kubernetes.io/projected/7243e221-39df-4cbe-bd32-302e9744977d-kube-api-access-9dltv\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:28:30.400505 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.400438 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h956r\" (UniqueName: \"kubernetes.io/projected/d4f98d41-0f79-4730-a09a-844c18aef0d4-kube-api-access-h956r\") pod \"console-operator-565b7b569d-4fgl4\" (UID: \"d4f98d41-0f79-4730-a09a-844c18aef0d4\") " pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:30.400692 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.400673 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5vwt\" (UniqueName: \"kubernetes.io/projected/2c70f851-1755-414a-afb0-f7c6ac1b38bf-kube-api-access-j5vwt\") pod \"network-check-source-7bc5b6bf85-4lxjv\" (UID: \"2c70f851-1755-414a-afb0-f7c6ac1b38bf\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4lxjv" Jul 22 02:28:30.400907 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.400885 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs4wv\" (UniqueName: \"kubernetes.io/projected/4a41203b-ba6d-4c90-a657-9c4cd9a748e9-kube-api-access-fs4wv\") pod \"ingress-canary-x2pkg\" (UID: \"4a41203b-ba6d-4c90-a657-9c4cd9a748e9\") " pod="openshift-ingress-canary/ingress-canary-x2pkg" Jul 22 02:28:30.412880 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.412847 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv" Jul 22 02:28:30.448079 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.448048 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:30.466226 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.466139 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qwwcl" Jul 22 02:28:30.492167 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.492140 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-66nws\" (UID: \"f1da4aa5-c459-4bfd-9a72-5e179d8af2e3\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-66nws" Jul 22 02:28:30.492291 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.492188 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-66nws\" (UID: \"f1da4aa5-c459-4bfd-9a72-5e179d8af2e3\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-66nws" Jul 22 02:28:30.492698 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.492348 2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 02:28:30.492698 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.492398 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert podName:f1da4aa5-c459-4bfd-9a72-5e179d8af2e3 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:30.99238373 +0000 UTC m=+35.841240543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-66nws" (UID: "f1da4aa5-c459-4bfd-9a72-5e179d8af2e3") : secret "networking-console-plugin-cert" not found Jul 22 02:28:30.492962 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.492921 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-66nws\" (UID: \"f1da4aa5-c459-4bfd-9a72-5e179d8af2e3\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-66nws" Jul 22 02:28:30.500430 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.500336 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4lxjv" Jul 22 02:28:30.524100 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.523792 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x2pkg" Jul 22 02:28:30.640859 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.640675 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:30.651344 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.649674 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-tq2tv"] Jul 22 02:28:30.662099 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.659043 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf"] Jul 22 02:28:30.664653 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:28:30.664596 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1239da74_1398_4bd0_9d28_98bacba6839a.slice/crio-5fdfa6c8afd8ca7f3f0a3b2ea57c83a3764702423394ca9858d40b2bafe49740 WatchSource:0}: Error finding container 5fdfa6c8afd8ca7f3f0a3b2ea57c83a3764702423394ca9858d40b2bafe49740: Status 404 returned error can't find the container with id 5fdfa6c8afd8ca7f3f0a3b2ea57c83a3764702423394ca9858d40b2bafe49740 Jul 22 02:28:30.677143 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.677110 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv"] Jul 22 02:28:30.694260 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:28:30.692005 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bf725ad_6951_48c4_b2e4_0d898eec59ed.slice/crio-ac774df6a7bcc90c1166579b6dac17244e08964e14912cdb1a4c4b51a2345309 WatchSource:0}: Error finding container ac774df6a7bcc90c1166579b6dac17244e08964e14912cdb1a4c4b51a2345309: Status 404 returned error can't find the container with id ac774df6a7bcc90c1166579b6dac17244e08964e14912cdb1a4c4b51a2345309 Jul 22 02:28:30.695158 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.695081 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-85lxl\" (UID: \"9f11a91e-5064-4838-bd58-7ab3369cc834\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" Jul 22 02:28:30.695284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.695206 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:30.695351 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.695337 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:28:30.695402 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.695354 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fd9dff84f-67px6: secret "image-registry-tls" not found Jul 22 02:28:30.695473 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.695403 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls podName:266db83d-8440-47b6-a7c7-8fb294a00554 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:31.695383241 +0000 UTC m=+36.544240068 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls") pod "image-registry-fd9dff84f-67px6" (UID: "266db83d-8440-47b6-a7c7-8fb294a00554") : secret "image-registry-tls" not found Jul 22 02:28:30.695529 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.695471 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:28:30.695529 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.695510 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls podName:9f11a91e-5064-4838-bd58-7ab3369cc834 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:31.695496142 +0000 UTC m=+36.544352952 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-85lxl" (UID: "9f11a91e-5064-4838-bd58-7ab3369cc834") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:28:30.708468 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.708446 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-4fgl4"] Jul 22 02:28:30.710749 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:28:30.710704 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4f98d41_0f79_4730_a09a_844c18aef0d4.slice/crio-4b9f3cb876284ba3274b550454599cbdbfaef5d3c22fb01cf4efb24cf1aae59a WatchSource:0}: Error finding container 4b9f3cb876284ba3274b550454599cbdbfaef5d3c22fb01cf4efb24cf1aae59a: Status 404 returned error can't find the container with id 4b9f3cb876284ba3274b550454599cbdbfaef5d3c22fb01cf4efb24cf1aae59a Jul 22 02:28:30.753816 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.752725 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:30.753816 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.753483 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qwwcl"] Jul 22 02:28:30.756121 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:28:30.756094 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d65fb9e_9e26_42f4_958d_67e85c7ddc31.slice/crio-85f78bd505ac15a6151d345f3e911da39d7175655a16865426a009dd46b382fc WatchSource:0}: Error finding container 85f78bd505ac15a6151d345f3e911da39d7175655a16865426a009dd46b382fc: Status 404 returned error can't find the container with id 85f78bd505ac15a6151d345f3e911da39d7175655a16865426a009dd46b382fc Jul 22 02:28:30.770736 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.770707 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x2pkg"] Jul 22 02:28:30.773517 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:28:30.773481 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a41203b_ba6d_4c90_a657_9c4cd9a748e9.slice/crio-d9dd9ae96e4889039962f72214a542468a60b49e1decc4b61d7fcc6e10bab6f9 WatchSource:0}: Error finding container d9dd9ae96e4889039962f72214a542468a60b49e1decc4b61d7fcc6e10bab6f9: Status 404 returned error can't find the container with id d9dd9ae96e4889039962f72214a542468a60b49e1decc4b61d7fcc6e10bab6f9 Jul 22 02:28:30.779795 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.779770 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-4lxjv"] Jul 22 02:28:30.782391 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:28:30.782369 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c70f851_1755_414a_afb0_f7c6ac1b38bf.slice/crio-3b40ccdedda96d2ace1202c0b8412cc5dfb4586fd16f5ce65075c096815ecfa4 WatchSource:0}: Error finding container 3b40ccdedda96d2ace1202c0b8412cc5dfb4586fd16f5ce65075c096815ecfa4: Status 404 returned error can't find the container with id 3b40ccdedda96d2ace1202c0b8412cc5dfb4586fd16f5ce65075c096815ecfa4 Jul 22 02:28:30.796096 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.796071 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gmxvw\" (UID: \"893b2a27-72a0-418c-9db2-2383934be8e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw" Jul 22 02:28:30.796243 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.796138 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:30.796243 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.796219 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:30.796323 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.796238 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 02:28:30.796323 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.796287 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle podName:5beca28a-e7a9-410d-ae0f-dce53b6faeae nodeName:}" failed. No retries permitted until 2026-07-22 02:28:31.796266258 +0000 UTC m=+36.645123077 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle") pod "router-default-5889846976-rpxs8" (UID: "5beca28a-e7a9-410d-ae0f-dce53b6faeae") : configmap references non-existent config key: service-ca.crt Jul 22 02:28:30.796408 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.796334 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:28:30.796408 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.796354 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls podName:893b2a27-72a0-418c-9db2-2383934be8e3 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:31.79633733 +0000 UTC m=+36.645194140 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gmxvw" (UID: "893b2a27-72a0-418c-9db2-2383934be8e3") : secret "samples-operator-tls" not found Jul 22 02:28:30.796408 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.796377 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs podName:5beca28a-e7a9-410d-ae0f-dce53b6faeae nodeName:}" failed. No retries permitted until 2026-07-22 02:28:31.796366144 +0000 UTC m=+36.645222955 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs") pod "router-default-5889846976-rpxs8" (UID: "5beca28a-e7a9-410d-ae0f-dce53b6faeae") : secret "router-metrics-certs-default" not found Jul 22 02:28:30.876284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.876245 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x2pkg" event={"ID":"4a41203b-ba6d-4c90-a657-9c4cd9a748e9","Type":"ContainerStarted","Data":"d9dd9ae96e4889039962f72214a542468a60b49e1decc4b61d7fcc6e10bab6f9"} Jul 22 02:28:30.879179 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.879153 2579 generic.go:356] "Generic (PLEG): container finished" podID="d18b98e3-c43e-44ff-8425-f19881254719" containerID="43f20f89e4dc234ac36f3dcea9b7b846be4a067131a81dc48dd9cb0acf6d4580" exitCode=0 Jul 22 02:28:30.879340 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.879267 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wpncq" event={"ID":"d18b98e3-c43e-44ff-8425-f19881254719","Type":"ContainerDied","Data":"43f20f89e4dc234ac36f3dcea9b7b846be4a067131a81dc48dd9cb0acf6d4580"} Jul 22 02:28:30.880375 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.880353 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qwwcl" event={"ID":"1d65fb9e-9e26-42f4-958d-67e85c7ddc31","Type":"ContainerStarted","Data":"85f78bd505ac15a6151d345f3e911da39d7175655a16865426a009dd46b382fc"} Jul 22 02:28:30.881341 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.881323 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" event={"ID":"d4f98d41-0f79-4730-a09a-844c18aef0d4","Type":"ContainerStarted","Data":"4b9f3cb876284ba3274b550454599cbdbfaef5d3c22fb01cf4efb24cf1aae59a"} Jul 22 02:28:30.882481 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.882461 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4lxjv" event={"ID":"2c70f851-1755-414a-afb0-f7c6ac1b38bf","Type":"ContainerStarted","Data":"3b40ccdedda96d2ace1202c0b8412cc5dfb4586fd16f5ce65075c096815ecfa4"} Jul 22 02:28:30.883337 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.883318 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv" event={"ID":"3bf725ad-6951-48c4-b2e4-0d898eec59ed","Type":"ContainerStarted","Data":"ac774df6a7bcc90c1166579b6dac17244e08964e14912cdb1a4c4b51a2345309"} Jul 22 02:28:30.884440 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.884420 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf" event={"ID":"1239da74-1398-4bd0-9d28-98bacba6839a","Type":"ContainerStarted","Data":"5fdfa6c8afd8ca7f3f0a3b2ea57c83a3764702423394ca9858d40b2bafe49740"} Jul 22 02:28:30.885497 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.885479 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-tq2tv" event={"ID":"858fe6a9-2e1c-48c7-8e67-12119323253e","Type":"ContainerStarted","Data":"e56aaa29f60227cba58d3b882ee2063aa3475e6fad2eb03dbd39664a37645774"} Jul 22 02:28:30.896824 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.896746 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:28:30.897021 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.896886 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:28:30.897021 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.896957 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls podName:7243e221-39df-4cbe-bd32-302e9744977d nodeName:}" failed. No retries permitted until 2026-07-22 02:28:31.896928535 +0000 UTC m=+36.745785348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls") pod "dns-default-6gfmw" (UID: "7243e221-39df-4cbe-bd32-302e9744977d") : secret "dns-default-metrics-tls" not found Jul 22 02:28:30.973352 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.973318 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:30.997401 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:30.997367 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-66nws\" (UID: \"f1da4aa5-c459-4bfd-9a72-5e179d8af2e3\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-66nws" Jul 22 02:28:30.997627 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.997588 2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 02:28:30.997744 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:30.997672 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert podName:f1da4aa5-c459-4bfd-9a72-5e179d8af2e3 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:31.997654564 +0000 UTC m=+36.846511394 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-66nws" (UID: "f1da4aa5-c459-4bfd-9a72-5e179d8af2e3") : secret "networking-console-plugin-cert" not found Jul 22 02:28:31.096635 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.096597 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:31.190187 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.190088 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:31.626416 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.622399 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:28:31.626416 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.622851 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:28:31.626416 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.623241 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:28:31.628920 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.627711 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-slk65\"" Jul 22 02:28:31.628920 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.627980 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 02:28:31.628920 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.628164 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kb24x\"" Jul 22 02:28:31.631234 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.631044 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 02:28:31.703899 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.703860 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-85lxl\" (UID: \"9f11a91e-5064-4838-bd58-7ab3369cc834\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" Jul 22 02:28:31.704101 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.704017 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:31.704213 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:31.704191 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:28:31.704274 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:31.704213 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fd9dff84f-67px6: secret "image-registry-tls" not found Jul 22 02:28:31.704274 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:31.704273 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls podName:266db83d-8440-47b6-a7c7-8fb294a00554 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:33.704253244 +0000 UTC m=+38.553110077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls") pod "image-registry-fd9dff84f-67px6" (UID: "266db83d-8440-47b6-a7c7-8fb294a00554") : secret "image-registry-tls" not found Jul 22 02:28:31.704721 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:31.704700 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:28:31.704807 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:31.704757 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls podName:9f11a91e-5064-4838-bd58-7ab3369cc834 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:33.704742103 +0000 UTC m=+38.553599120 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-85lxl" (UID: "9f11a91e-5064-4838-bd58-7ab3369cc834") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:28:31.787889 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.787796 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:31.805210 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.805175 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gmxvw\" (UID: \"893b2a27-72a0-418c-9db2-2383934be8e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw" Jul 22 02:28:31.805400 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.805259 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:31.805400 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.805335 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:31.805510 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:31.805478 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:28:31.805568 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:31.805539 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs podName:5beca28a-e7a9-410d-ae0f-dce53b6faeae nodeName:}" failed. No retries permitted until 2026-07-22 02:28:33.805520263 +0000 UTC m=+38.654377074 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs") pod "router-default-5889846976-rpxs8" (UID: "5beca28a-e7a9-410d-ae0f-dce53b6faeae") : secret "router-metrics-certs-default" not found Jul 22 02:28:31.806443 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:31.805747 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle podName:5beca28a-e7a9-410d-ae0f-dce53b6faeae nodeName:}" failed. No retries permitted until 2026-07-22 02:28:33.805728679 +0000 UTC m=+38.654585492 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle") pod "router-default-5889846976-rpxs8" (UID: "5beca28a-e7a9-410d-ae0f-dce53b6faeae") : configmap references non-existent config key: service-ca.crt Jul 22 02:28:31.806443 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:31.805831 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 02:28:31.806443 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:31.805871 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls podName:893b2a27-72a0-418c-9db2-2383934be8e3 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:33.805858258 +0000 UTC m=+38.654715072 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gmxvw" (UID: "893b2a27-72a0-418c-9db2-2383934be8e3") : secret "samples-operator-tls" not found Jul 22 02:28:31.888079 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.887998 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:31.897214 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.897125 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wpncq" event={"ID":"d18b98e3-c43e-44ff-8425-f19881254719","Type":"ContainerStarted","Data":"2dfd489a0b355413472e333868bee61658bfd7683ac14cb7b7fd858813f928df"} Jul 22 02:28:31.906234 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.906200 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:28:31.906410 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:31.906342 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:28:31.906410 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:31.906400 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls podName:7243e221-39df-4cbe-bd32-302e9744977d nodeName:}" failed. No retries permitted until 2026-07-22 02:28:33.90638161 +0000 UTC m=+38.755238434 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls") pod "dns-default-6gfmw" (UID: "7243e221-39df-4cbe-bd32-302e9744977d") : secret "dns-default-metrics-tls" not found Jul 22 02:28:31.925421 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:31.925355 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wpncq" podStartSLOduration=8.355132942000001 podStartE2EDuration="36.925332732s" totalImagesPullingTime="28.57019979s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:27:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:31.92370894 +0000 UTC m=+36.772565774" watchObservedRunningTime="2026-07-22 02:28:31.925332732 +0000 UTC m=+36.774189566" Jul 22 02:28:32.006705 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:32.006657 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-66nws\" (UID: \"f1da4aa5-c459-4bfd-9a72-5e179d8af2e3\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-66nws" Jul 22 02:28:32.007482 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:32.007458 2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 02:28:32.007596 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:32.007530 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert podName:f1da4aa5-c459-4bfd-9a72-5e179d8af2e3 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:34.007507972 +0000 UTC m=+38.856364787 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-66nws" (UID: "f1da4aa5-c459-4bfd-9a72-5e179d8af2e3") : secret "networking-console-plugin-cert" not found Jul 22 02:28:32.193652 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:32.193548 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:32.260152 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:32.260084 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:32.992918 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:32.992714 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:33.056507 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:33.056458 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:33.392785 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:33.392746 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:33.493616 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:33.493579 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:33.726404 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:33.726108 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:33.726642 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:33.726273 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:28:33.726642 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:33.726446 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fd9dff84f-67px6: secret "image-registry-tls" not found Jul 22 02:28:33.726642 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:33.726455 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-85lxl\" (UID: \"9f11a91e-5064-4838-bd58-7ab3369cc834\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" Jul 22 02:28:33.726642 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:33.726511 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls podName:266db83d-8440-47b6-a7c7-8fb294a00554 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:37.726490659 +0000 UTC m=+42.575347480 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls") pod "image-registry-fd9dff84f-67px6" (UID: "266db83d-8440-47b6-a7c7-8fb294a00554") : secret "image-registry-tls" not found Jul 22 02:28:33.726642 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:33.726553 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:28:33.726642 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:33.726626 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls podName:9f11a91e-5064-4838-bd58-7ab3369cc834 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:37.726610215 +0000 UTC m=+42.575467029 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-85lxl" (UID: "9f11a91e-5064-4838-bd58-7ab3369cc834") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:28:33.827934 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:33.827895 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:33.828132 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:33.828048 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gmxvw\" (UID: \"893b2a27-72a0-418c-9db2-2383934be8e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw" Jul 22 02:28:33.828132 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:33.828109 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:33.828242 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:33.828167 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:28:33.828242 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:33.828214 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 02:28:33.828242 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:33.828241 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs podName:5beca28a-e7a9-410d-ae0f-dce53b6faeae nodeName:}" failed. No retries permitted until 2026-07-22 02:28:37.828221056 +0000 UTC m=+42.677077878 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs") pod "router-default-5889846976-rpxs8" (UID: "5beca28a-e7a9-410d-ae0f-dce53b6faeae") : secret "router-metrics-certs-default" not found Jul 22 02:28:33.828389 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:33.828266 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls podName:893b2a27-72a0-418c-9db2-2383934be8e3 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:37.828252219 +0000 UTC m=+42.677109045 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gmxvw" (UID: "893b2a27-72a0-418c-9db2-2383934be8e3") : secret "samples-operator-tls" not found Jul 22 02:28:33.828389 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:33.828280 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle podName:5beca28a-e7a9-410d-ae0f-dce53b6faeae nodeName:}" failed. No retries permitted until 2026-07-22 02:28:37.828274721 +0000 UTC m=+42.677131531 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle") pod "router-default-5889846976-rpxs8" (UID: "5beca28a-e7a9-410d-ae0f-dce53b6faeae") : configmap references non-existent config key: service-ca.crt Jul 22 02:28:33.929468 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:33.929431 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:28:33.929671 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:33.929583 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:28:33.929671 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:33.929665 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls podName:7243e221-39df-4cbe-bd32-302e9744977d nodeName:}" failed. No retries permitted until 2026-07-22 02:28:37.929644509 +0000 UTC m=+42.778501340 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls") pod "dns-default-6gfmw" (UID: "7243e221-39df-4cbe-bd32-302e9744977d") : secret "dns-default-metrics-tls" not found Jul 22 02:28:34.030248 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:34.030165 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-66nws\" (UID: \"f1da4aa5-c459-4bfd-9a72-5e179d8af2e3\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-66nws" Jul 22 02:28:34.030644 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:34.030317 2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 02:28:34.030644 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:34.030386 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert podName:f1da4aa5-c459-4bfd-9a72-5e179d8af2e3 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:38.030372077 +0000 UTC m=+42.879228888 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-66nws" (UID: "f1da4aa5-c459-4bfd-9a72-5e179d8af2e3") : secret "networking-console-plugin-cert" not found Jul 22 02:28:35.422742 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:35.422704 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:35.483402 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:35.483368 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:37.555356 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:37.555322 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:37.621766 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:37.621735 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:37.765268 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:37.765232 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:37.765452 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:37.765336 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-85lxl\" (UID: \"9f11a91e-5064-4838-bd58-7ab3369cc834\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" Jul 22 02:28:37.765452 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:37.765393 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:28:37.765452 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:37.765415 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fd9dff84f-67px6: secret "image-registry-tls" not found Jul 22 02:28:37.765600 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:37.765472 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:28:37.765600 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:37.765486 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls podName:266db83d-8440-47b6-a7c7-8fb294a00554 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:45.765465923 +0000 UTC m=+50.614322738 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls") pod "image-registry-fd9dff84f-67px6" (UID: "266db83d-8440-47b6-a7c7-8fb294a00554") : secret "image-registry-tls" not found Jul 22 02:28:37.765600 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:37.765517 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls podName:9f11a91e-5064-4838-bd58-7ab3369cc834 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:45.765505319 +0000 UTC m=+50.614362131 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-85lxl" (UID: "9f11a91e-5064-4838-bd58-7ab3369cc834") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:28:37.866829 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:37.866783 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gmxvw\" (UID: \"893b2a27-72a0-418c-9db2-2383934be8e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw" Jul 22 02:28:37.867129 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:37.866855 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:37.867129 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:37.866908 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:37.867129 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:37.866933 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 02:28:37.867129 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:37.867037 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls podName:893b2a27-72a0-418c-9db2-2383934be8e3 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:45.86701786 +0000 UTC m=+50.715874670 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gmxvw" (UID: "893b2a27-72a0-418c-9db2-2383934be8e3") : secret "samples-operator-tls" not found Jul 22 02:28:37.867129 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:37.867053 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle podName:5beca28a-e7a9-410d-ae0f-dce53b6faeae nodeName:}" failed. No retries permitted until 2026-07-22 02:28:45.867047487 +0000 UTC m=+50.715904297 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle") pod "router-default-5889846976-rpxs8" (UID: "5beca28a-e7a9-410d-ae0f-dce53b6faeae") : configmap references non-existent config key: service-ca.crt Jul 22 02:28:37.867129 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:37.867101 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:28:37.867365 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:37.867150 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs podName:5beca28a-e7a9-410d-ae0f-dce53b6faeae nodeName:}" failed. No retries permitted until 2026-07-22 02:28:45.867138271 +0000 UTC m=+50.715995085 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs") pod "router-default-5889846976-rpxs8" (UID: "5beca28a-e7a9-410d-ae0f-dce53b6faeae") : secret "router-metrics-certs-default" not found Jul 22 02:28:37.968250 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:37.968203 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:28:37.968409 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:37.968352 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:28:37.968448 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:37.968414 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls podName:7243e221-39df-4cbe-bd32-302e9744977d nodeName:}" failed. No retries permitted until 2026-07-22 02:28:45.968398596 +0000 UTC m=+50.817255411 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls") pod "dns-default-6gfmw" (UID: "7243e221-39df-4cbe-bd32-302e9744977d") : secret "dns-default-metrics-tls" not found Jul 22 02:28:38.068852 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:38.068815 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-66nws\" (UID: \"f1da4aa5-c459-4bfd-9a72-5e179d8af2e3\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-66nws" Jul 22 02:28:38.069025 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:38.068985 2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 02:28:38.069068 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:38.069054 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert podName:f1da4aa5-c459-4bfd-9a72-5e179d8af2e3 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:46.069035436 +0000 UTC m=+50.917892247 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-66nws" (UID: "f1da4aa5-c459-4bfd-9a72-5e179d8af2e3") : secret "networking-console-plugin-cert" not found Jul 22 02:28:38.396563 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:38.396536 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:38.497593 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:38.497568 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:38.922398 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:38.921901 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-tq2tv" event={"ID":"858fe6a9-2e1c-48c7-8e67-12119323253e","Type":"ContainerStarted","Data":"c6e6100d12767c56eb328a7271348ef3b5c857f4511a37efff243ca61dc19255"} Jul 22 02:28:38.923490 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:38.923465 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x2pkg" event={"ID":"4a41203b-ba6d-4c90-a657-9c4cd9a748e9","Type":"ContainerStarted","Data":"f1fc67cc125fb90532e098ad7ff81b49ca71fc9ed186f1e3c4d830fb0b12f552"} Jul 22 02:28:38.925163 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:38.925095 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qwwcl" event={"ID":"1d65fb9e-9e26-42f4-958d-67e85c7ddc31","Type":"ContainerStarted","Data":"fa9b2630f29bb8277c77c219e9786b51b159003a9643c65a6430c63871e397dd"} Jul 22 02:28:38.926509 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:38.926489 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/0.log" Jul 22 02:28:38.926610 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:38.926526 2579 generic.go:356] "Generic (PLEG): container finished" podID="d4f98d41-0f79-4730-a09a-844c18aef0d4" containerID="92435ef9808580540f0c982fec7b1964ffb019f626347fcad2e611c71df2027d" exitCode=255 Jul 22 02:28:38.926610 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:38.926556 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" event={"ID":"d4f98d41-0f79-4730-a09a-844c18aef0d4","Type":"ContainerDied","Data":"92435ef9808580540f0c982fec7b1964ffb019f626347fcad2e611c71df2027d"} Jul 22 02:28:38.926792 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:38.926765 2579 scope.go:117] "RemoveContainer" containerID="92435ef9808580540f0c982fec7b1964ffb019f626347fcad2e611c71df2027d" Jul 22 02:28:38.929149 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:38.928027 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4lxjv" event={"ID":"2c70f851-1755-414a-afb0-f7c6ac1b38bf","Type":"ContainerStarted","Data":"f8e1875db69636a5a6939e40948c027538d48f31de6ff804068b2466f3eed0fc"} Jul 22 02:28:38.929819 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:38.929801 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv" event={"ID":"3bf725ad-6951-48c4-b2e4-0d898eec59ed","Type":"ContainerStarted","Data":"c72a2446f8295a32251012d4fb877b64497a32a2b97bc57817cfab230c955191"} Jul 22 02:28:38.931553 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:38.931272 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf" event={"ID":"1239da74-1398-4bd0-9d28-98bacba6839a","Type":"ContainerStarted","Data":"836b0973f4f173e1a7ed437374b9a07b7b88d45957f7a5f870beddf3ba0ff1a3"} Jul 22 02:28:38.940925 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:38.940612 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-tq2tv" podStartSLOduration=29.313664605 podStartE2EDuration="36.940596543s" totalImagesPullingTime="7.626931938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:38.940229567 +0000 UTC m=+43.789086401" watchObservedRunningTime="2026-07-22 02:28:38.940596543 +0000 UTC m=+43.789453378" Jul 22 02:28:38.982049 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:38.981333 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qwwcl" podStartSLOduration=28.501250047 podStartE2EDuration="35.981310937s" totalImagesPullingTime="7.48006089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:38.959225986 +0000 UTC m=+43.808082807" watchObservedRunningTime="2026-07-22 02:28:38.981310937 +0000 UTC m=+43.830167771" Jul 22 02:28:38.982049 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:38.981627 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4lxjv" podStartSLOduration=28.471600306 podStartE2EDuration="35.981614991s" totalImagesPullingTime="7.510014685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:38.981248959 +0000 UTC m=+43.830105791" watchObservedRunningTime="2026-07-22 02:28:38.981614991 +0000 UTC m=+43.830471823" Jul 22 02:28:39.026803 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.026755 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv" podStartSLOduration=28.436198121 podStartE2EDuration="36.026737759s" totalImagesPullingTime="7.590539638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:39.025353845 +0000 UTC m=+43.874210677" watchObservedRunningTime="2026-07-22 02:28:39.026737759 +0000 UTC m=+43.875594591" Jul 22 02:28:39.047408 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.047346 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-x2pkg" podStartSLOduration=2.537502123 podStartE2EDuration="10.04732299s" totalImagesPullingTime="7.509820867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:39.045758922 +0000 UTC m=+43.894615754" watchObservedRunningTime="2026-07-22 02:28:39.04732299 +0000 UTC m=+43.896179823" Jul 22 02:28:39.072639 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.072513 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:39.146959 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.146907 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:39.752889 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.752797 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:39.800892 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.800829 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf" podStartSLOduration=29.183750635 podStartE2EDuration="36.800807051s" totalImagesPullingTime="7.617056416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:39.069434602 +0000 UTC m=+43.918291433" watchObservedRunningTime="2026-07-22 02:28:39.800807051 +0000 UTC m=+44.649663865" Jul 22 02:28:39.802699 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.801996 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-v44rl"] Jul 22 02:28:39.820128 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.820096 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-v44rl"] Jul 22 02:28:39.820286 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.820230 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-v44rl" Jul 22 02:28:39.823143 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.823114 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 22 02:28:39.824027 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.824012 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ftn2p\"" Jul 22 02:28:39.824095 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.824025 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 22 02:28:39.843287 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.843248 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:39.939044 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.939014 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 02:28:39.939504 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.939453 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/0.log" Jul 22 02:28:39.939504 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.939487 2579 generic.go:356] "Generic (PLEG): container finished" podID="d4f98d41-0f79-4730-a09a-844c18aef0d4" containerID="0eabe667ea6daa66e791e9c80d66a60a101a0f819b1fe52fe1d31536dbfd9960" exitCode=255 Jul 22 02:28:39.939605 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.939527 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" event={"ID":"d4f98d41-0f79-4730-a09a-844c18aef0d4","Type":"ContainerDied","Data":"0eabe667ea6daa66e791e9c80d66a60a101a0f819b1fe52fe1d31536dbfd9960"} Jul 22 02:28:39.939656 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.939617 2579 scope.go:117] "RemoveContainer" containerID="92435ef9808580540f0c982fec7b1964ffb019f626347fcad2e611c71df2027d" Jul 22 02:28:39.939802 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.939784 2579 scope.go:117] "RemoveContainer" containerID="0eabe667ea6daa66e791e9c80d66a60a101a0f819b1fe52fe1d31536dbfd9960" Jul 22 02:28:39.940008 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:39.939986 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-4fgl4_openshift-console-operator(d4f98d41-0f79-4730-a09a-844c18aef0d4)\"" pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" podUID="d4f98d41-0f79-4730-a09a-844c18aef0d4" Jul 22 02:28:39.983907 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:39.983881 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpxlk\" (UniqueName: \"kubernetes.io/projected/a68a9d1b-bcef-479b-a21d-45eaca9c7a40-kube-api-access-rpxlk\") pod \"migrator-7cc9c7ffd7-v44rl\" (UID: \"a68a9d1b-bcef-479b-a21d-45eaca9c7a40\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-v44rl" Jul 22 02:28:40.010269 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.010189 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:40.085400 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.085345 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpxlk\" (UniqueName: \"kubernetes.io/projected/a68a9d1b-bcef-479b-a21d-45eaca9c7a40-kube-api-access-rpxlk\") pod \"migrator-7cc9c7ffd7-v44rl\" (UID: \"a68a9d1b-bcef-479b-a21d-45eaca9c7a40\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-v44rl" Jul 22 02:28:40.113215 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.113142 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpxlk\" (UniqueName: \"kubernetes.io/projected/a68a9d1b-bcef-479b-a21d-45eaca9c7a40-kube-api-access-rpxlk\") pod \"migrator-7cc9c7ffd7-v44rl\" (UID: \"a68a9d1b-bcef-479b-a21d-45eaca9c7a40\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-v44rl" Jul 22 02:28:40.124996 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.124959 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:40.129634 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.129611 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-v44rl" Jul 22 02:28:40.133622 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.133595 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8dnzm"] Jul 22 02:28:40.164500 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.164408 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8dnzm"] Jul 22 02:28:40.164636 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.164504 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:40.169470 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.168070 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 02:28:40.169470 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.168296 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 02:28:40.169470 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.168475 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-btmmd\"" Jul 22 02:28:40.252364 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.252331 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:40.287551 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.287472 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:40.287551 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.287550 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f7165bc-e726-4334-8e57-49c05f994762-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:40.287723 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.287605 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f7165bc-e726-4334-8e57-49c05f994762-crio-socket\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:40.287723 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.287634 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f7165bc-e726-4334-8e57-49c05f994762-data-volume\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:40.287723 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.287689 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq69v\" (UniqueName: \"kubernetes.io/projected/7f7165bc-e726-4334-8e57-49c05f994762-kube-api-access-hq69v\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:40.316384 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.316299 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-v44rl"] Jul 22 02:28:40.318775 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:28:40.318746 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda68a9d1b_bcef_479b_a21d_45eaca9c7a40.slice/crio-7c3db282199ad91b95a13c0f7fbf117280f28c156b5be4a86aea53202dd5b4b3 WatchSource:0}: Error finding container 7c3db282199ad91b95a13c0f7fbf117280f28c156b5be4a86aea53202dd5b4b3: Status 404 returned error can't find the container with id 7c3db282199ad91b95a13c0f7fbf117280f28c156b5be4a86aea53202dd5b4b3 Jul 22 02:28:40.389167 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.389135 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f7165bc-e726-4334-8e57-49c05f994762-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:40.389343 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.389181 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f7165bc-e726-4334-8e57-49c05f994762-crio-socket\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:40.389343 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.389206 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f7165bc-e726-4334-8e57-49c05f994762-data-volume\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:40.389343 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.389246 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq69v\" (UniqueName: \"kubernetes.io/projected/7f7165bc-e726-4334-8e57-49c05f994762-kube-api-access-hq69v\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:40.389343 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:40.389267 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:28:40.389546 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.389349 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:40.389546 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:40.389372 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7f7165bc-e726-4334-8e57-49c05f994762-kube-rbac-proxy-cm podName:7f7165bc-e726-4334-8e57-49c05f994762 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:40.889352688 +0000 UTC m=+45.738209503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7f7165bc-e726-4334-8e57-49c05f994762-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8dnzm" (UID: "7f7165bc-e726-4334-8e57-49c05f994762") : configmap "kube-rbac-proxy" not found Jul 22 02:28:40.389546 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.389269 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f7165bc-e726-4334-8e57-49c05f994762-crio-socket\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:40.389546 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:40.389430 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:28:40.389546 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:40.389480 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls podName:7f7165bc-e726-4334-8e57-49c05f994762 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:40.889463145 +0000 UTC m=+45.738319958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8dnzm" (UID: "7f7165bc-e726-4334-8e57-49c05f994762") : secret "insights-runtime-extractor-tls" not found Jul 22 02:28:40.389546 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.389501 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f7165bc-e726-4334-8e57-49c05f994762-data-volume\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:40.404056 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.404028 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq69v\" (UniqueName: \"kubernetes.io/projected/7f7165bc-e726-4334-8e57-49c05f994762-kube-api-access-hq69v\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:40.448147 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.448119 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:40.448312 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.448181 2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:40.473510 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.473472 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:40.576666 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.576644 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:40.894412 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.894324 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:40.894579 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.894432 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f7165bc-e726-4334-8e57-49c05f994762-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:40.894579 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:40.894515 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:28:40.894579 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:40.894527 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:28:40.894739 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:40.894588 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls podName:7f7165bc-e726-4334-8e57-49c05f994762 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:41.894572259 +0000 UTC m=+46.743429069 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8dnzm" (UID: "7f7165bc-e726-4334-8e57-49c05f994762") : secret "insights-runtime-extractor-tls" not found Jul 22 02:28:40.894739 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:40.894604 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7f7165bc-e726-4334-8e57-49c05f994762-kube-rbac-proxy-cm podName:7f7165bc-e726-4334-8e57-49c05f994762 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:41.894597929 +0000 UTC m=+46.743454740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7f7165bc-e726-4334-8e57-49c05f994762-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8dnzm" (UID: "7f7165bc-e726-4334-8e57-49c05f994762") : configmap "kube-rbac-proxy" not found Jul 22 02:28:40.943907 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.943870 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-v44rl" event={"ID":"a68a9d1b-bcef-479b-a21d-45eaca9c7a40","Type":"ContainerStarted","Data":"7c3db282199ad91b95a13c0f7fbf117280f28c156b5be4a86aea53202dd5b4b3"} Jul 22 02:28:40.945467 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.945448 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 02:28:40.945976 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:40.945809 2579 scope.go:117] "RemoveContainer" containerID="0eabe667ea6daa66e791e9c80d66a60a101a0f819b1fe52fe1d31536dbfd9960" Jul 22 02:28:40.946055 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:40.946008 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-4fgl4_openshift-console-operator(d4f98d41-0f79-4730-a09a-844c18aef0d4)\"" pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" podUID="d4f98d41-0f79-4730-a09a-844c18aef0d4" Jul 22 02:28:41.904739 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:41.904705 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f7165bc-e726-4334-8e57-49c05f994762-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:41.904927 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:41.904840 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:28:41.904927 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:41.904854 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:41.904927 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:41.904899 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7f7165bc-e726-4334-8e57-49c05f994762-kube-rbac-proxy-cm podName:7f7165bc-e726-4334-8e57-49c05f994762 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:43.904881864 +0000 UTC m=+48.753738686 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7f7165bc-e726-4334-8e57-49c05f994762-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8dnzm" (UID: "7f7165bc-e726-4334-8e57-49c05f994762") : configmap "kube-rbac-proxy" not found Jul 22 02:28:41.905172 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:41.904996 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:28:41.905172 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:41.905062 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls podName:7f7165bc-e726-4334-8e57-49c05f994762 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:43.905045403 +0000 UTC m=+48.753902235 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8dnzm" (UID: "7f7165bc-e726-4334-8e57-49c05f994762") : secret "insights-runtime-extractor-tls" not found Jul 22 02:28:41.948997 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:41.948975 2579 scope.go:117] "RemoveContainer" containerID="0eabe667ea6daa66e791e9c80d66a60a101a0f819b1fe52fe1d31536dbfd9960" Jul 22 02:28:41.949291 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:41.949124 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-4fgl4_openshift-console-operator(d4f98d41-0f79-4730-a09a-844c18aef0d4)\"" pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" podUID="d4f98d41-0f79-4730-a09a-844c18aef0d4" Jul 22 02:28:42.077040 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.077006 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-g62nd"] Jul 22 02:28:42.112437 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.112414 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-g62nd"] Jul 22 02:28:42.112536 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.112504 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-g62nd" Jul 22 02:28:42.115452 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.115432 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 22 02:28:42.115558 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.115466 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 22 02:28:42.115754 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.115740 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 22 02:28:42.116601 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.116584 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 22 02:28:42.116691 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.116584 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-cvqtf\"" Jul 22 02:28:42.125573 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.125549 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:42.208254 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.208165 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjdgh\" (UniqueName: \"kubernetes.io/projected/3c869eaa-c33b-4e5c-9157-b9bb57006a4c-kube-api-access-hjdgh\") pod \"service-ca-58598695f7-g62nd\" (UID: \"3c869eaa-c33b-4e5c-9157-b9bb57006a4c\") " pod="openshift-service-ca/service-ca-58598695f7-g62nd" Jul 22 02:28:42.208254 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.208206 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3c869eaa-c33b-4e5c-9157-b9bb57006a4c-signing-key\") pod \"service-ca-58598695f7-g62nd\" (UID: \"3c869eaa-c33b-4e5c-9157-b9bb57006a4c\") " pod="openshift-service-ca/service-ca-58598695f7-g62nd" Jul 22 02:28:42.208254 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.208223 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3c869eaa-c33b-4e5c-9157-b9bb57006a4c-signing-cabundle\") pod \"service-ca-58598695f7-g62nd\" (UID: \"3c869eaa-c33b-4e5c-9157-b9bb57006a4c\") " pod="openshift-service-ca/service-ca-58598695f7-g62nd" Jul 22 02:28:42.236214 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.236179 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:42.309598 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.309568 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjdgh\" (UniqueName: \"kubernetes.io/projected/3c869eaa-c33b-4e5c-9157-b9bb57006a4c-kube-api-access-hjdgh\") pod \"service-ca-58598695f7-g62nd\" (UID: \"3c869eaa-c33b-4e5c-9157-b9bb57006a4c\") " pod="openshift-service-ca/service-ca-58598695f7-g62nd" Jul 22 02:28:42.309598 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.309600 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3c869eaa-c33b-4e5c-9157-b9bb57006a4c-signing-key\") pod \"service-ca-58598695f7-g62nd\" (UID: \"3c869eaa-c33b-4e5c-9157-b9bb57006a4c\") " pod="openshift-service-ca/service-ca-58598695f7-g62nd" Jul 22 02:28:42.309782 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.309623 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3c869eaa-c33b-4e5c-9157-b9bb57006a4c-signing-cabundle\") pod \"service-ca-58598695f7-g62nd\" (UID: \"3c869eaa-c33b-4e5c-9157-b9bb57006a4c\") " pod="openshift-service-ca/service-ca-58598695f7-g62nd" Jul 22 02:28:42.310376 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.310355 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3c869eaa-c33b-4e5c-9157-b9bb57006a4c-signing-cabundle\") pod \"service-ca-58598695f7-g62nd\" (UID: \"3c869eaa-c33b-4e5c-9157-b9bb57006a4c\") " pod="openshift-service-ca/service-ca-58598695f7-g62nd" Jul 22 02:28:42.310495 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.310480 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vzq9c_21af75d3-4d8f-49b7-b96f-db2dc2cd5275/dns-node-resolver/0.log" Jul 22 02:28:42.312169 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.312147 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3c869eaa-c33b-4e5c-9157-b9bb57006a4c-signing-key\") pod \"service-ca-58598695f7-g62nd\" (UID: \"3c869eaa-c33b-4e5c-9157-b9bb57006a4c\") " pod="openshift-service-ca/service-ca-58598695f7-g62nd" Jul 22 02:28:42.323524 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.323493 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjdgh\" (UniqueName: \"kubernetes.io/projected/3c869eaa-c33b-4e5c-9157-b9bb57006a4c-kube-api-access-hjdgh\") pod \"service-ca-58598695f7-g62nd\" (UID: \"3c869eaa-c33b-4e5c-9157-b9bb57006a4c\") " pod="openshift-service-ca/service-ca-58598695f7-g62nd" Jul 22 02:28:42.421530 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.421495 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-g62nd" Jul 22 02:28:42.579473 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.579364 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-g62nd"] Jul 22 02:28:42.581984 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:28:42.581954 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c869eaa_c33b_4e5c_9157_b9bb57006a4c.slice/crio-619bc306f97181b096d91240c7a366f7779a211f71a0f28b820a9d6b2cc96513 WatchSource:0}: Error finding container 619bc306f97181b096d91240c7a366f7779a211f71a0f28b820a9d6b2cc96513: Status 404 returned error can't find the container with id 619bc306f97181b096d91240c7a366f7779a211f71a0f28b820a9d6b2cc96513 Jul 22 02:28:42.912514 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.912492 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fwbqh_7369bc37-a20a-44e7-9af0-2b15ac190fce/node-ca/0.log" Jul 22 02:28:42.953294 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.953259 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-v44rl" event={"ID":"a68a9d1b-bcef-479b-a21d-45eaca9c7a40","Type":"ContainerStarted","Data":"af7ab6468163663acefa13128404035aa78e9c046ac585d335ddf19b1bbcbb7d"} Jul 22 02:28:42.953705 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.953329 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-v44rl" event={"ID":"a68a9d1b-bcef-479b-a21d-45eaca9c7a40","Type":"ContainerStarted","Data":"765fe86de13f80fba32082d44209b065f98ecf15749609f17b029d39bd0480ae"} Jul 22 02:28:42.954680 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.954656 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-g62nd" event={"ID":"3c869eaa-c33b-4e5c-9157-b9bb57006a4c","Type":"ContainerStarted","Data":"87c6c38d710ac00891cfd183d6aefba4921a44a067014328674937b278d53bc0"} Jul 22 02:28:42.954787 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.954699 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-g62nd" event={"ID":"3c869eaa-c33b-4e5c-9157-b9bb57006a4c","Type":"ContainerStarted","Data":"619bc306f97181b096d91240c7a366f7779a211f71a0f28b820a9d6b2cc96513"} Jul 22 02:28:42.975598 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.975539 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-v44rl" podStartSLOduration=2.417509672 podStartE2EDuration="3.975521004s" totalImagesPullingTime="1.558011332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:42.975317496 +0000 UTC m=+47.824174328" watchObservedRunningTime="2026-07-22 02:28:42.975521004 +0000 UTC m=+47.824377838" Jul 22 02:28:42.993142 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:42.993094 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-g62nd" podStartSLOduration=0.993074073 podStartE2EDuration="993.074073ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:42.991812484 +0000 UTC m=+47.840669316" watchObservedRunningTime="2026-07-22 02:28:42.993074073 +0000 UTC m=+47.841930904" Jul 22 02:28:43.350211 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:43.350174 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:43.711242 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:43.711163 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-x2pkg_4a41203b-ba6d-4c90-a657-9c4cd9a748e9/serve-healthcheck-canary/0.log" Jul 22 02:28:43.909504 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:43.909475 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-v44rl_a68a9d1b-bcef-479b-a21d-45eaca9c7a40/migrator/0.log" Jul 22 02:28:43.925382 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:43.925351 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f7165bc-e726-4334-8e57-49c05f994762-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:43.925545 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:43.925454 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:43.925545 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:43.925480 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:28:43.925545 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:43.925540 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7f7165bc-e726-4334-8e57-49c05f994762-kube-rbac-proxy-cm podName:7f7165bc-e726-4334-8e57-49c05f994762 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:47.925525736 +0000 UTC m=+52.774382549 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7f7165bc-e726-4334-8e57-49c05f994762-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8dnzm" (UID: "7f7165bc-e726-4334-8e57-49c05f994762") : configmap "kube-rbac-proxy" not found Jul 22 02:28:43.925675 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:43.925572 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:28:43.925675 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:43.925611 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls podName:7f7165bc-e726-4334-8e57-49c05f994762 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:47.925600971 +0000 UTC m=+52.774457781 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8dnzm" (UID: "7f7165bc-e726-4334-8e57-49c05f994762") : secret "insights-runtime-extractor-tls" not found Jul 22 02:28:44.120443 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:44.120395 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:44.124506 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:44.124484 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-v44rl_a68a9d1b-bcef-479b-a21d-45eaca9c7a40/graceful-termination/0.log" Jul 22 02:28:44.214916 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:44.214887 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:44.312267 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:44.312240 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-924jv_3bf725ad-6951-48c4-b2e4-0d898eec59ed/kube-storage-version-migrator-operator/0.log" Jul 22 02:28:44.312485 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:44.312458 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:44.414358 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:44.414264 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:44.483431 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:44.483382 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:44.815969 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:44.815865 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:45.845096 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:45.845057 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:28:45.845476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:45.845176 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-85lxl\" (UID: \"9f11a91e-5064-4838-bd58-7ab3369cc834\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" Jul 22 02:28:45.845476 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:45.845239 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:28:45.845476 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:45.845267 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fd9dff84f-67px6: secret "image-registry-tls" not found Jul 22 02:28:45.845476 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:45.845303 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:28:45.845476 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:45.845354 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls podName:266db83d-8440-47b6-a7c7-8fb294a00554 nodeName:}" failed. No retries permitted until 2026-07-22 02:29:01.845332772 +0000 UTC m=+66.694189655 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls") pod "image-registry-fd9dff84f-67px6" (UID: "266db83d-8440-47b6-a7c7-8fb294a00554") : secret "image-registry-tls" not found Jul 22 02:28:45.845476 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:45.845370 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls podName:9f11a91e-5064-4838-bd58-7ab3369cc834 nodeName:}" failed. No retries permitted until 2026-07-22 02:29:01.845363594 +0000 UTC m=+66.694220404 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-85lxl" (UID: "9f11a91e-5064-4838-bd58-7ab3369cc834") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:28:45.946349 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:45.946310 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gmxvw\" (UID: \"893b2a27-72a0-418c-9db2-2383934be8e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw" Jul 22 02:28:45.946541 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:45.946388 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:45.946541 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:45.946443 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:28:45.946637 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:45.946556 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle podName:5beca28a-e7a9-410d-ae0f-dce53b6faeae nodeName:}" failed. No retries permitted until 2026-07-22 02:29:01.94653508 +0000 UTC m=+66.795391895 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle") pod "router-default-5889846976-rpxs8" (UID: "5beca28a-e7a9-410d-ae0f-dce53b6faeae") : configmap references non-existent config key: service-ca.crt Jul 22 02:28:45.946637 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:45.946612 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:28:45.946743 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:45.946674 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs podName:5beca28a-e7a9-410d-ae0f-dce53b6faeae nodeName:}" failed. No retries permitted until 2026-07-22 02:29:01.946658728 +0000 UTC m=+66.795515542 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs") pod "router-default-5889846976-rpxs8" (UID: "5beca28a-e7a9-410d-ae0f-dce53b6faeae") : secret "router-metrics-certs-default" not found Jul 22 02:28:45.949031 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:45.948999 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/893b2a27-72a0-418c-9db2-2383934be8e3-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gmxvw\" (UID: \"893b2a27-72a0-418c-9db2-2383934be8e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw" Jul 22 02:28:46.003415 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:46.003366 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:46.025106 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:46.025075 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw" Jul 22 02:28:46.047635 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:46.047606 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:28:46.047793 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:46.047761 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:28:46.047835 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:46.047825 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls podName:7243e221-39df-4cbe-bd32-302e9744977d nodeName:}" failed. No retries permitted until 2026-07-22 02:29:02.047810507 +0000 UTC m=+66.896667321 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls") pod "dns-default-6gfmw" (UID: "7243e221-39df-4cbe-bd32-302e9744977d") : secret "dns-default-metrics-tls" not found Jul 22 02:28:46.148750 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:46.148712 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-66nws\" (UID: \"f1da4aa5-c459-4bfd-9a72-5e179d8af2e3\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-66nws" Jul 22 02:28:46.148932 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:46.148868 2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 02:28:46.149016 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:46.148978 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert podName:f1da4aa5-c459-4bfd-9a72-5e179d8af2e3 nodeName:}" failed. No retries permitted until 2026-07-22 02:29:02.148930131 +0000 UTC m=+66.997786944 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-66nws" (UID: "f1da4aa5-c459-4bfd-9a72-5e179d8af2e3") : secret "networking-console-plugin-cert" not found Jul 22 02:28:46.174972 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:46.174918 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw"] Jul 22 02:28:46.198079 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:46.198035 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:46.391066 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:46.390976 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:46.574281 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:46.574240 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:46.642074 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:46.641984 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:46.802918 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:46.802882 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:46.970747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:46.970666 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw" event={"ID":"893b2a27-72a0-418c-9db2-2383934be8e3","Type":"ContainerStarted","Data":"45c50c46249241b4a19e0c9bb0500c3d295c95ecad9b99743bd9c4080a3be1e7"} Jul 22 02:28:47.282341 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:47.282251 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:47.843533 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:47.843479 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:47.966271 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:47.966237 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:47.966444 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:47.966327 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f7165bc-e726-4334-8e57-49c05f994762-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:47.966444 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:47.966392 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:28:47.966521 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:28:47.966459 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls podName:7f7165bc-e726-4334-8e57-49c05f994762 nodeName:}" failed. No retries permitted until 2026-07-22 02:28:55.966444091 +0000 UTC m=+60.815300901 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8dnzm" (UID: "7f7165bc-e726-4334-8e57-49c05f994762") : secret "insights-runtime-extractor-tls" not found Jul 22 02:28:47.966842 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:47.966825 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f7165bc-e726-4334-8e57-49c05f994762-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:48.054639 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:48.054593 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:48.195304 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:48.195219 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:48.320077 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:48.320046 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:48.457293 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:48.457262 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:48.661715 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:48.661682 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:48.762351 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:48.762260 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:48.866159 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:48.866123 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:48.979208 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:48.979178 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw" event={"ID":"893b2a27-72a0-418c-9db2-2383934be8e3","Type":"ContainerStarted","Data":"93514ea3b7af366ae21682e91bc04277d1328d3534d7318f46a4e3f55065fe87"} Jul 22 02:28:48.979388 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:48.979237 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw" event={"ID":"893b2a27-72a0-418c-9db2-2383934be8e3","Type":"ContainerStarted","Data":"f0d56b3b62227b6c641f5ca401941c077ca2def04aedeb7bf235ef103b178224"} Jul 22 02:28:49.019836 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:49.019738 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gmxvw" podStartSLOduration=43.906821275 podStartE2EDuration="46.019722331s" totalImagesPullingTime="2.112901056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:49.018866146 +0000 UTC m=+53.867722979" watchObservedRunningTime="2026-07-22 02:28:49.019722331 +0000 UTC m=+53.868579161" Jul 22 02:28:49.787873 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:49.787839 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:50.003769 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:50.003741 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:50.204999 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:50.204968 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:50.392580 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:50.392543 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:50.599103 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:50.599070 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:50.781903 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:50.781864 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:51.605995 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:51.605957 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:52.184996 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:52.184959 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:52.390032 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:52.389997 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:52.786728 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:52.786688 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:52.981573 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:52.981536 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:53.430142 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:53.430107 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:53.506844 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:53.506809 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:53.622717 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:53.622685 2579 scope.go:117] "RemoveContainer" containerID="0eabe667ea6daa66e791e9c80d66a60a101a0f819b1fe52fe1d31536dbfd9960" Jul 22 02:28:53.987149 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:53.987118 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:53.998311 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:53.998288 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 02:28:53.998481 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:53.998376 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" event={"ID":"d4f98d41-0f79-4730-a09a-844c18aef0d4","Type":"ContainerStarted","Data":"c23c4c85df0842438bbcb2c4dac36aba79e15bbdad6a15e3353f6484973fc912"} Jul 22 02:28:53.998755 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:53.998734 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:54.022279 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:54.022230 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" podStartSLOduration=43.450632609 podStartE2EDuration="51.022214859s" totalImagesPullingTime="7.57158225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:54.021763924 +0000 UTC m=+58.870620757" watchObservedRunningTime="2026-07-22 02:28:54.022214859 +0000 UTC m=+58.871071690" Jul 22 02:28:54.100618 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:54.100591 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-4fgl4" Jul 22 02:28:54.278573 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:54.278491 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:54.381932 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:54.381883 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:54.482568 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:54.482539 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:54.593044 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:54.593013 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:54.653305 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:54.653269 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:54.782527 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:54.782497 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:54.883575 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:54.883487 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:54.983372 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:54.983338 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:55.053771 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:55.053736 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:55.365576 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:55.365540 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:55.463724 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:55.463687 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:55.568266 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:55.568229 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:55.637150 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:55.637059 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:55.871198 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:55.871172 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nl7s9" Jul 22 02:28:55.971981 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:55.971878 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:56.033406 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:56.033362 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:56.035693 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:56.035668 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f7165bc-e726-4334-8e57-49c05f994762-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8dnzm\" (UID: \"7f7165bc-e726-4334-8e57-49c05f994762\") " pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:56.083494 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:56.083465 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-btmmd\"" Jul 22 02:28:56.091409 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:56.091383 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8dnzm" Jul 22 02:28:56.183695 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:56.183659 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:56.244353 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:56.244273 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8dnzm"] Jul 22 02:28:56.247739 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:28:56.247713 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f7165bc_e726_4334_8e57_49c05f994762.slice/crio-d341fe499cef1b1666df87c6747a83abb1fb80bfbd5a233172b388ac21a9e0b8 WatchSource:0}: Error finding container d341fe499cef1b1666df87c6747a83abb1fb80bfbd5a233172b388ac21a9e0b8: Status 404 returned error can't find the container with id d341fe499cef1b1666df87c6747a83abb1fb80bfbd5a233172b388ac21a9e0b8 Jul 22 02:28:56.832561 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:56.832427 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:56.983041 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:56.982931 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:28:57.010638 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:57.010604 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8dnzm" event={"ID":"7f7165bc-e726-4334-8e57-49c05f994762","Type":"ContainerStarted","Data":"d341fe499cef1b1666df87c6747a83abb1fb80bfbd5a233172b388ac21a9e0b8"} Jul 22 02:28:57.083745 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:28:57.083662 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:01.286327 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.286283 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs\") pod \"network-metrics-daemon-48qrg\" (UID: \"ddc83329-034a-42bd-a8b3-9bf395c15e9b\") " pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:29:01.289159 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.289138 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 02:29:01.299522 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.299495 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddc83329-034a-42bd-a8b3-9bf395c15e9b-metrics-certs\") pod \"network-metrics-daemon-48qrg\" (UID: \"ddc83329-034a-42bd-a8b3-9bf395c15e9b\") " pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:29:01.365730 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.365695 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-slk65\"" Jul 22 02:29:01.373344 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.373316 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48qrg" Jul 22 02:29:01.386819 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.386784 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret\") pod \"global-pull-secret-syncer-cjr5p\" (UID: \"72d84003-0a34-4663-be5f-09d9d7a14498\") " pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:29:01.389620 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.389600 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 02:29:01.400338 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.400307 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72d84003-0a34-4663-be5f-09d9d7a14498-original-pull-secret\") pod \"global-pull-secret-syncer-cjr5p\" (UID: \"72d84003-0a34-4663-be5f-09d9d7a14498\") " pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:29:01.488205 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.488166 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhvdt\" (UniqueName: \"kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt\") pod \"network-check-target-9kvjj\" (UID: \"97e803aa-bf79-4624-b82d-417735a543a4\") " pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:29:01.490988 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.490962 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhvdt\" (UniqueName: \"kubernetes.io/projected/97e803aa-bf79-4624-b82d-417735a543a4-kube-api-access-vhvdt\") pod \"network-check-target-9kvjj\" (UID: \"97e803aa-bf79-4624-b82d-417735a543a4\") " pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:29:01.646090 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.646063 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kb24x\"" Jul 22 02:29:01.653837 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.653813 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:29:01.653964 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.653836 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr5p" Jul 22 02:29:01.754743 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.754694 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-48qrg"] Jul 22 02:29:01.822468 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.822438 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9kvjj"] Jul 22 02:29:01.825764 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:29:01.825738 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97e803aa_bf79_4624_b82d_417735a543a4.slice/crio-e620eb178955d47c01fa9a8da2abec9da4a7e65a1a3ca7ea0a47c18d25c9fa2e WatchSource:0}: Error finding container e620eb178955d47c01fa9a8da2abec9da4a7e65a1a3ca7ea0a47c18d25c9fa2e: Status 404 returned error can't find the container with id e620eb178955d47c01fa9a8da2abec9da4a7e65a1a3ca7ea0a47c18d25c9fa2e Jul 22 02:29:01.837878 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.837855 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cjr5p"] Jul 22 02:29:01.839934 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:29:01.839912 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72d84003_0a34_4663_be5f_09d9d7a14498.slice/crio-18d266534eac69ca86d9d0efca6e1446282b3a894e1a02eb6629b587991f714c WatchSource:0}: Error finding container 18d266534eac69ca86d9d0efca6e1446282b3a894e1a02eb6629b587991f714c: Status 404 returned error can't find the container with id 18d266534eac69ca86d9d0efca6e1446282b3a894e1a02eb6629b587991f714c Jul 22 02:29:01.893451 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.893409 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:29:01.893615 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.893490 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-85lxl\" (UID: \"9f11a91e-5064-4838-bd58-7ab3369cc834\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" Jul 22 02:29:01.896007 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.895982 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f11a91e-5064-4838-bd58-7ab3369cc834-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-85lxl\" (UID: \"9f11a91e-5064-4838-bd58-7ab3369cc834\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" Jul 22 02:29:01.896007 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.895999 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls\") pod \"image-registry-fd9dff84f-67px6\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:29:01.994230 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.994197 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:29:01.994230 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.994239 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:29:01.994869 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.994810 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5beca28a-e7a9-410d-ae0f-dce53b6faeae-service-ca-bundle\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:29:01.996695 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:01.996673 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5beca28a-e7a9-410d-ae0f-dce53b6faeae-metrics-certs\") pod \"router-default-5889846976-rpxs8\" (UID: \"5beca28a-e7a9-410d-ae0f-dce53b6faeae\") " pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:29:02.030642 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.030597 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8dnzm" event={"ID":"7f7165bc-e726-4334-8e57-49c05f994762","Type":"ContainerStarted","Data":"db77952fab6b78ffc073ca1428cb38dd0fc94f59576878d86360a4a33ce856d0"} Jul 22 02:29:02.031778 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.031751 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cjr5p" event={"ID":"72d84003-0a34-4663-be5f-09d9d7a14498","Type":"ContainerStarted","Data":"18d266534eac69ca86d9d0efca6e1446282b3a894e1a02eb6629b587991f714c"} Jul 22 02:29:02.033074 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.033052 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9kvjj" event={"ID":"97e803aa-bf79-4624-b82d-417735a543a4","Type":"ContainerStarted","Data":"46d871c4cd163d56e2458e46983dd8ec0da792610e57151b4dbc7247e5a6e1c0"} Jul 22 02:29:02.033190 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.033089 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9kvjj" event={"ID":"97e803aa-bf79-4624-b82d-417735a543a4","Type":"ContainerStarted","Data":"e620eb178955d47c01fa9a8da2abec9da4a7e65a1a3ca7ea0a47c18d25c9fa2e"} Jul 22 02:29:02.033190 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.033141 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:29:02.034150 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.034133 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-48qrg" event={"ID":"ddc83329-034a-42bd-a8b3-9bf395c15e9b","Type":"ContainerStarted","Data":"c85b011b96cc84b3a6ba88822caaccb50ab8d496a5428188b9a5d43a83aa076c"} Jul 22 02:29:02.048503 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.048449 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9kvjj" podStartSLOduration=66.048432203 podStartE2EDuration="1m6.048432203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:27:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:02.047540915 +0000 UTC m=+66.896397748" watchObservedRunningTime="2026-07-22 02:29:02.048432203 +0000 UTC m=+66.897289035" Jul 22 02:29:02.095352 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.095311 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:29:02.097704 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.097684 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7243e221-39df-4cbe-bd32-302e9744977d-metrics-tls\") pod \"dns-default-6gfmw\" (UID: \"7243e221-39df-4cbe-bd32-302e9744977d\") " pod="openshift-dns/dns-default-6gfmw" Jul 22 02:29:02.115537 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.115488 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-48hxd\"" Jul 22 02:29:02.123002 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.122925 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:29:02.178748 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.178681 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dnrd4\"" Jul 22 02:29:02.186019 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.185991 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" Jul 22 02:29:02.195992 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.195717 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-66nws\" (UID: \"f1da4aa5-c459-4bfd-9a72-5e179d8af2e3\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-66nws" Jul 22 02:29:02.198548 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.198514 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1da4aa5-c459-4bfd-9a72-5e179d8af2e3-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-66nws\" (UID: \"f1da4aa5-c459-4bfd-9a72-5e179d8af2e3\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-66nws" Jul 22 02:29:02.211221 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.210999 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2z4h4\"" Jul 22 02:29:02.218707 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.218682 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:29:02.286610 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.286273 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2lw7w\"" Jul 22 02:29:02.292195 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.291919 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-fd9dff84f-67px6"] Jul 22 02:29:02.292625 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.292137 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6gfmw" Jul 22 02:29:02.297647 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:29:02.297622 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod266db83d_8440_47b6_a7c7_8fb294a00554.slice/crio-a06429bed930a5e2dd9372a7cd463eb84f1e5afc06f04942c61e575210661b3e WatchSource:0}: Error finding container a06429bed930a5e2dd9372a7cd463eb84f1e5afc06f04942c61e575210661b3e: Status 404 returned error can't find the container with id a06429bed930a5e2dd9372a7cd463eb84f1e5afc06f04942c61e575210661b3e Jul 22 02:29:02.386575 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.386334 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2s967\"" Jul 22 02:29:02.394342 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.393692 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-66nws" Jul 22 02:29:02.400196 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.400118 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl"] Jul 22 02:29:02.413120 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:29:02.413084 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f11a91e_5064_4838_bd58_7ab3369cc834.slice/crio-95b9eb959acbd599b88cf008fc9c5988ff564377af08c9c3cc0b5b776d7b6bba WatchSource:0}: Error finding container 95b9eb959acbd599b88cf008fc9c5988ff564377af08c9c3cc0b5b776d7b6bba: Status 404 returned error can't find the container with id 95b9eb959acbd599b88cf008fc9c5988ff564377af08c9c3cc0b5b776d7b6bba Jul 22 02:29:02.440910 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.440817 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5889846976-rpxs8"] Jul 22 02:29:02.500981 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.500823 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6gfmw"] Jul 22 02:29:02.611496 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:02.611372 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-66nws"] Jul 22 02:29:02.619140 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:29:02.619099 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1da4aa5_c459_4bfd_9a72_5e179d8af2e3.slice/crio-59307b933561a44769c872bc9246be50f02f977a728ef73ffc84130098b3e486 WatchSource:0}: Error finding container 59307b933561a44769c872bc9246be50f02f977a728ef73ffc84130098b3e486: Status 404 returned error can't find the container with id 59307b933561a44769c872bc9246be50f02f977a728ef73ffc84130098b3e486 Jul 22 02:29:03.043300 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:03.043156 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-66nws" event={"ID":"f1da4aa5-c459-4bfd-9a72-5e179d8af2e3","Type":"ContainerStarted","Data":"59307b933561a44769c872bc9246be50f02f977a728ef73ffc84130098b3e486"} Jul 22 02:29:03.047981 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:03.047198 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fd9dff84f-67px6" event={"ID":"266db83d-8440-47b6-a7c7-8fb294a00554","Type":"ContainerStarted","Data":"49720997cc38ded0e37fc10c1384868f149976b2084f9cdc7261b8355347e588"} Jul 22 02:29:03.047981 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:03.047285 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fd9dff84f-67px6" event={"ID":"266db83d-8440-47b6-a7c7-8fb294a00554","Type":"ContainerStarted","Data":"a06429bed930a5e2dd9372a7cd463eb84f1e5afc06f04942c61e575210661b3e"} Jul 22 02:29:03.048197 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:03.048024 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:29:03.050543 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:03.050518 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5889846976-rpxs8" event={"ID":"5beca28a-e7a9-410d-ae0f-dce53b6faeae","Type":"ContainerStarted","Data":"92e72ea56afb93971b431b2fa405de9692f0a3d6e250ac527d7fbe9b2a97df3c"} Jul 22 02:29:03.050663 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:03.050561 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5889846976-rpxs8" event={"ID":"5beca28a-e7a9-410d-ae0f-dce53b6faeae","Type":"ContainerStarted","Data":"67df539a7e048b3bac91b0f21e8363ae180378c8965476e322d8c0fcc48f85c1"} Jul 22 02:29:03.052969 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:03.052195 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" event={"ID":"9f11a91e-5064-4838-bd58-7ab3369cc834","Type":"ContainerStarted","Data":"95b9eb959acbd599b88cf008fc9c5988ff564377af08c9c3cc0b5b776d7b6bba"} Jul 22 02:29:03.055990 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:03.055917 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6gfmw" event={"ID":"7243e221-39df-4cbe-bd32-302e9744977d","Type":"ContainerStarted","Data":"ffacac1e4e71b429a2c3a0703d8a7b8dc37b3559770db45a25552c11db448c40"} Jul 22 02:29:03.071287 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:03.071032 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-fd9dff84f-67px6" podStartSLOduration=67.071017061 podStartE2EDuration="1m7.071017061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:27:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:03.068606915 +0000 UTC m=+67.917463753" watchObservedRunningTime="2026-07-22 02:29:03.071017061 +0000 UTC m=+67.919873893" Jul 22 02:29:03.088749 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:03.088672 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5889846976-rpxs8" podStartSLOduration=61.088650585 podStartE2EDuration="1m1.088650585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:03.086650203 +0000 UTC m=+67.935507034" watchObservedRunningTime="2026-07-22 02:29:03.088650585 +0000 UTC m=+67.937507420" Jul 22 02:29:03.219227 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:03.219166 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:29:03.221991 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:03.221963 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:29:04.065777 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:04.065735 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-48qrg" event={"ID":"ddc83329-034a-42bd-a8b3-9bf395c15e9b","Type":"ContainerStarted","Data":"e3d4bc418cf82d26c46131a8f29d2e953639f88470107e44cb38643a863d30c9"} Jul 22 02:29:04.066318 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:04.065804 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-48qrg" event={"ID":"ddc83329-034a-42bd-a8b3-9bf395c15e9b","Type":"ContainerStarted","Data":"83e986cfe975bba2edba07f140e83e602dacabb3cf3ab103f81650d0f3525f12"} Jul 22 02:29:04.066844 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:04.066808 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:29:04.069009 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:04.068987 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5889846976-rpxs8" Jul 22 02:29:04.106311 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:04.106252 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-48qrg" podStartSLOduration=67.550221253 podStartE2EDuration="1m9.106234217s" totalImagesPullingTime="1.556012964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:04.095778644 +0000 UTC m=+68.944635505" watchObservedRunningTime="2026-07-22 02:29:04.106234217 +0000 UTC m=+68.955091049" Jul 22 02:29:05.758471 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.758408 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g"] Jul 22 02:29:05.768545 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.768514 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq"] Jul 22 02:29:05.768714 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.768691 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.774412 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.774384 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 22 02:29:05.774589 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.774470 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 22 02:29:05.775547 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.775524 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 22 02:29:05.775970 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.775712 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 22 02:29:05.775970 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.775571 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 22 02:29:05.775970 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.775825 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 22 02:29:05.777558 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.776906 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 22 02:29:05.777558 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.777123 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq" Jul 22 02:29:05.783305 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.783284 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 22 02:29:05.797659 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.797630 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57c45b69d8-f5hj6"] Jul 22 02:29:05.805720 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.805671 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq"] Jul 22 02:29:05.805864 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.805812 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57c45b69d8-f5hj6" Jul 22 02:29:05.808802 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.808772 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g"] Jul 22 02:29:05.811051 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.811032 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-72scw\"" Jul 22 02:29:05.811304 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.811285 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 22 02:29:05.821156 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.821130 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57c45b69d8-f5hj6"] Jul 22 02:29:05.832125 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.832091 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d68cb4ff-40dc-4020-bac3-0db866ecf648-tmp\") pod \"klusterlet-addon-workmgr-749bd4fb67-bzfxq\" (UID: \"d68cb4ff-40dc-4020-bac3-0db866ecf648\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq" Jul 22 02:29:05.832382 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.832361 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/145b0768-765c-48a5-85f6-c55397a4fe0d-hub\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.832491 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.832475 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66v2q\" (UniqueName: \"kubernetes.io/projected/d68cb4ff-40dc-4020-bac3-0db866ecf648-kube-api-access-66v2q\") pod \"klusterlet-addon-workmgr-749bd4fb67-bzfxq\" (UID: \"d68cb4ff-40dc-4020-bac3-0db866ecf648\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq" Jul 22 02:29:05.832586 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.832575 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7xln\" (UniqueName: \"kubernetes.io/projected/145b0768-765c-48a5-85f6-c55397a4fe0d-kube-api-access-z7xln\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.832690 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.832677 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d68cb4ff-40dc-4020-bac3-0db866ecf648-klusterlet-config\") pod \"klusterlet-addon-workmgr-749bd4fb67-bzfxq\" (UID: \"d68cb4ff-40dc-4020-bac3-0db866ecf648\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq" Jul 22 02:29:05.832799 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.832785 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/145b0768-765c-48a5-85f6-c55397a4fe0d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.832927 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.832912 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/145b0768-765c-48a5-85f6-c55397a4fe0d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.833092 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.833078 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/145b0768-765c-48a5-85f6-c55397a4fe0d-ca\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.833204 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.833191 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/145b0768-765c-48a5-85f6-c55397a4fe0d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.883488 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.883449 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-phhmt"] Jul 22 02:29:05.888118 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.888083 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cb8d48668-9mpmn"] Jul 22 02:29:05.888284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.888234 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-phhmt" Jul 22 02:29:05.891195 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.890913 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-fsvzh\"" Jul 22 02:29:05.891195 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.890972 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 22 02:29:05.891195 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.891057 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 22 02:29:05.891392 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.891273 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:05.894730 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.894710 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 02:29:05.894858 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.894714 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-q7fvk\"" Jul 22 02:29:05.895037 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.895019 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 02:29:05.895143 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.895125 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 02:29:05.895143 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.895126 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 02:29:05.895252 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.895226 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 02:29:05.903915 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.903876 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cb8d48668-9mpmn"] Jul 22 02:29:05.905576 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.905550 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-phhmt"] Jul 22 02:29:05.933897 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.933818 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66v2q\" (UniqueName: \"kubernetes.io/projected/d68cb4ff-40dc-4020-bac3-0db866ecf648-kube-api-access-66v2q\") pod \"klusterlet-addon-workmgr-749bd4fb67-bzfxq\" (UID: \"d68cb4ff-40dc-4020-bac3-0db866ecf648\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq" Jul 22 02:29:05.933897 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.933861 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58bw4\" (UniqueName: \"kubernetes.io/projected/2500f8f3-355d-47ef-af40-da1b1e9a6ccd-kube-api-access-58bw4\") pod \"downloads-5bc67dcd46-phhmt\" (UID: \"2500f8f3-355d-47ef-af40-da1b1e9a6ccd\") " pod="openshift-console/downloads-5bc67dcd46-phhmt" Jul 22 02:29:05.934112 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.933907 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-oauth-serving-cert\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:05.934112 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.933962 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7xln\" (UniqueName: \"kubernetes.io/projected/145b0768-765c-48a5-85f6-c55397a4fe0d-kube-api-access-z7xln\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.934112 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.933997 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f13956b7-8c9e-4a56-8bd9-8b7acf0af176-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57c45b69d8-f5hj6\" (UID: \"f13956b7-8c9e-4a56-8bd9-8b7acf0af176\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57c45b69d8-f5hj6" Jul 22 02:29:05.934112 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.934025 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-oauth-config\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:05.934112 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.934072 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d68cb4ff-40dc-4020-bac3-0db866ecf648-klusterlet-config\") pod \"klusterlet-addon-workmgr-749bd4fb67-bzfxq\" (UID: \"d68cb4ff-40dc-4020-bac3-0db866ecf648\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq" Jul 22 02:29:05.934112 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.934102 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/145b0768-765c-48a5-85f6-c55397a4fe0d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.934345 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.934123 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-service-ca\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:05.934345 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.934149 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-serving-cert\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:05.934345 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.934182 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/145b0768-765c-48a5-85f6-c55397a4fe0d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.934345 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.934218 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/145b0768-765c-48a5-85f6-c55397a4fe0d-ca\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.934345 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.934248 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/145b0768-765c-48a5-85f6-c55397a4fe0d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.934345 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.934273 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2gkq\" (UniqueName: \"kubernetes.io/projected/f13956b7-8c9e-4a56-8bd9-8b7acf0af176-kube-api-access-n2gkq\") pod \"managed-serviceaccount-addon-agent-57c45b69d8-f5hj6\" (UID: \"f13956b7-8c9e-4a56-8bd9-8b7acf0af176\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57c45b69d8-f5hj6" Jul 22 02:29:05.934345 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.934311 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d68cb4ff-40dc-4020-bac3-0db866ecf648-tmp\") pod \"klusterlet-addon-workmgr-749bd4fb67-bzfxq\" (UID: \"d68cb4ff-40dc-4020-bac3-0db866ecf648\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq" Jul 22 02:29:05.934345 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.934332 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrv7p\" (UniqueName: \"kubernetes.io/projected/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-kube-api-access-hrv7p\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:05.934719 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.934378 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-config\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:05.934719 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.934402 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/145b0768-765c-48a5-85f6-c55397a4fe0d-hub\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.935138 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.934976 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d68cb4ff-40dc-4020-bac3-0db866ecf648-tmp\") pod \"klusterlet-addon-workmgr-749bd4fb67-bzfxq\" (UID: \"d68cb4ff-40dc-4020-bac3-0db866ecf648\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq" Jul 22 02:29:05.935138 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.935078 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/145b0768-765c-48a5-85f6-c55397a4fe0d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.937387 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.937362 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/145b0768-765c-48a5-85f6-c55397a4fe0d-ca\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.937621 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.937599 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d68cb4ff-40dc-4020-bac3-0db866ecf648-klusterlet-config\") pod \"klusterlet-addon-workmgr-749bd4fb67-bzfxq\" (UID: \"d68cb4ff-40dc-4020-bac3-0db866ecf648\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq" Jul 22 02:29:05.937708 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.937693 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/145b0768-765c-48a5-85f6-c55397a4fe0d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.937789 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.937758 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/145b0768-765c-48a5-85f6-c55397a4fe0d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.938317 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.938293 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/145b0768-765c-48a5-85f6-c55397a4fe0d-hub\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.950075 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.949981 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:05.951372 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.951350 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7xln\" (UniqueName: \"kubernetes.io/projected/145b0768-765c-48a5-85f6-c55397a4fe0d-kube-api-access-z7xln\") pod \"cluster-proxy-proxy-agent-7875bd8555-56v2g\" (UID: \"145b0768-765c-48a5-85f6-c55397a4fe0d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:05.952098 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:05.952074 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66v2q\" (UniqueName: \"kubernetes.io/projected/d68cb4ff-40dc-4020-bac3-0db866ecf648-kube-api-access-66v2q\") pod \"klusterlet-addon-workmgr-749bd4fb67-bzfxq\" (UID: \"d68cb4ff-40dc-4020-bac3-0db866ecf648\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq" Jul 22 02:29:06.033437 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.033397 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:06.035465 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.035443 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-service-ca\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:06.035627 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.035476 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-serving-cert\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:06.035627 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.035525 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2gkq\" (UniqueName: \"kubernetes.io/projected/f13956b7-8c9e-4a56-8bd9-8b7acf0af176-kube-api-access-n2gkq\") pod \"managed-serviceaccount-addon-agent-57c45b69d8-f5hj6\" (UID: \"f13956b7-8c9e-4a56-8bd9-8b7acf0af176\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57c45b69d8-f5hj6" Jul 22 02:29:06.035627 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.035556 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrv7p\" (UniqueName: \"kubernetes.io/projected/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-kube-api-access-hrv7p\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:06.035627 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.035598 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-config\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:06.035824 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.035627 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58bw4\" (UniqueName: \"kubernetes.io/projected/2500f8f3-355d-47ef-af40-da1b1e9a6ccd-kube-api-access-58bw4\") pod \"downloads-5bc67dcd46-phhmt\" (UID: \"2500f8f3-355d-47ef-af40-da1b1e9a6ccd\") " pod="openshift-console/downloads-5bc67dcd46-phhmt" Jul 22 02:29:06.035824 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.035652 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-oauth-serving-cert\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:06.035824 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.035683 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f13956b7-8c9e-4a56-8bd9-8b7acf0af176-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57c45b69d8-f5hj6\" (UID: \"f13956b7-8c9e-4a56-8bd9-8b7acf0af176\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57c45b69d8-f5hj6" Jul 22 02:29:06.035824 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.035708 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-oauth-config\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:06.036257 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.036231 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-service-ca\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:06.036408 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.036367 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-oauth-serving-cert\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:06.036598 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.036556 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-config\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:06.038448 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.038425 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f13956b7-8c9e-4a56-8bd9-8b7acf0af176-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57c45b69d8-f5hj6\" (UID: \"f13956b7-8c9e-4a56-8bd9-8b7acf0af176\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57c45b69d8-f5hj6" Jul 22 02:29:06.038544 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.038519 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-oauth-config\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:06.038646 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.038592 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-serving-cert\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:06.048880 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.048851 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2gkq\" (UniqueName: \"kubernetes.io/projected/f13956b7-8c9e-4a56-8bd9-8b7acf0af176-kube-api-access-n2gkq\") pod \"managed-serviceaccount-addon-agent-57c45b69d8-f5hj6\" (UID: \"f13956b7-8c9e-4a56-8bd9-8b7acf0af176\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57c45b69d8-f5hj6" Jul 22 02:29:06.049050 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.048891 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58bw4\" (UniqueName: \"kubernetes.io/projected/2500f8f3-355d-47ef-af40-da1b1e9a6ccd-kube-api-access-58bw4\") pod \"downloads-5bc67dcd46-phhmt\" (UID: \"2500f8f3-355d-47ef-af40-da1b1e9a6ccd\") " pod="openshift-console/downloads-5bc67dcd46-phhmt" Jul 22 02:29:06.049328 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.049308 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrv7p\" (UniqueName: \"kubernetes.io/projected/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-kube-api-access-hrv7p\") pod \"console-5cb8d48668-9mpmn\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:06.107759 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.107728 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" Jul 22 02:29:06.116562 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.116537 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq" Jul 22 02:29:06.124665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.124644 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57c45b69d8-f5hj6" Jul 22 02:29:06.129627 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.129594 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:06.216212 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.216122 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-phhmt" Jul 22 02:29:06.220542 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.220515 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:06.330658 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.330612 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:06.430775 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:06.430737 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:07.016484 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.016444 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d7b6cf945-r86gn"] Jul 22 02:29:07.025644 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.025615 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.037072 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.037043 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d7b6cf945-r86gn"] Jul 22 02:29:07.037826 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.037798 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:07.038047 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.038031 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 02:29:07.104517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.104479 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:07.146549 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.146466 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-trusted-ca-bundle\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.146549 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.146529 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-oauth-serving-cert\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.146767 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.146568 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-oauth-config\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.146767 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.146620 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-service-ca\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.146767 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.146642 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5mzm\" (UniqueName: \"kubernetes.io/projected/b6f86fa5-28b5-4a34-a437-28756a33fdf4-kube-api-access-d5mzm\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.146767 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.146672 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-serving-cert\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.146919 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.146784 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-config\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.247505 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.247470 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-oauth-serving-cert\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.247694 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.247521 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-oauth-config\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.247694 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.247563 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-service-ca\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.247694 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.247583 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5mzm\" (UniqueName: \"kubernetes.io/projected/b6f86fa5-28b5-4a34-a437-28756a33fdf4-kube-api-access-d5mzm\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.247694 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.247611 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-serving-cert\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.247694 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.247671 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-config\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.247984 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.247705 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-trusted-ca-bundle\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.248446 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.248416 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-oauth-serving-cert\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.248556 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.248537 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-trusted-ca-bundle\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.248556 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.248544 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-config\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.249081 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.249033 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-service-ca\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.250643 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.250623 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-serving-cert\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.250899 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.250877 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-oauth-config\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.259097 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.259059 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5mzm\" (UniqueName: \"kubernetes.io/projected/b6f86fa5-28b5-4a34-a437-28756a33fdf4-kube-api-access-d5mzm\") pod \"console-d7b6cf945-r86gn\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:07.339638 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:07.339585 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:08.207649 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:08.207603 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:08.269594 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:08.269565 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:08.401419 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:08.401386 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:08.465359 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:08.465223 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:08.723808 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:08.723722 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:08.829624 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:08.829589 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:08.917674 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:08.917643 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:09.582423 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:09.582357 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq"] Jul 22 02:29:09.588261 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:09.588058 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-phhmt"] Jul 22 02:29:09.661198 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:09.661162 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d7b6cf945-r86gn"] Jul 22 02:29:09.681060 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:09.679843 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g"] Jul 22 02:29:09.691624 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:29:09.691552 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd68cb4ff_40dc_4020_bac3_0db866ecf648.slice/crio-47724a836c0a5e0b7ddbf6b1a8e3c45c0a50cb2036472856bc5af9324ceafae3 WatchSource:0}: Error finding container 47724a836c0a5e0b7ddbf6b1a8e3c45c0a50cb2036472856bc5af9324ceafae3: Status 404 returned error can't find the container with id 47724a836c0a5e0b7ddbf6b1a8e3c45c0a50cb2036472856bc5af9324ceafae3 Jul 22 02:29:09.692598 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:29:09.692377 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2500f8f3_355d_47ef_af40_da1b1e9a6ccd.slice/crio-6d3862b90120f11248906a1cae8f04d313197ab9b682642fe1596c3e1f4a900c WatchSource:0}: Error finding container 6d3862b90120f11248906a1cae8f04d313197ab9b682642fe1596c3e1f4a900c: Status 404 returned error can't find the container with id 6d3862b90120f11248906a1cae8f04d313197ab9b682642fe1596c3e1f4a900c Jul 22 02:29:09.694745 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:29:09.694723 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f86fa5_28b5_4a34_a437_28756a33fdf4.slice/crio-20e3b3bc40dc171abe9c538c521703c9ead865061bc740d7d6b0ff45357c3244 WatchSource:0}: Error finding container 20e3b3bc40dc171abe9c538c521703c9ead865061bc740d7d6b0ff45357c3244: Status 404 returned error can't find the container with id 20e3b3bc40dc171abe9c538c521703c9ead865061bc740d7d6b0ff45357c3244 Jul 22 02:29:09.694881 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:29:09.694868 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod145b0768_765c_48a5_85f6_c55397a4fe0d.slice/crio-5ed10b63146fa9310b09342e4fd549f6784dd3638fc9555f7c1b7f1d483cc222 WatchSource:0}: Error finding container 5ed10b63146fa9310b09342e4fd549f6784dd3638fc9555f7c1b7f1d483cc222: Status 404 returned error can't find the container with id 5ed10b63146fa9310b09342e4fd549f6784dd3638fc9555f7c1b7f1d483cc222 Jul 22 02:29:09.818962 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:09.818911 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57c45b69d8-f5hj6"] Jul 22 02:29:09.824259 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:29:09.823053 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf13956b7_8c9e_4a56_8bd9_8b7acf0af176.slice/crio-0299f70a1dc486c61a3fe6fb3c8f9fa35f50cd736e9734056b3806b1744d9d93 WatchSource:0}: Error finding container 0299f70a1dc486c61a3fe6fb3c8f9fa35f50cd736e9734056b3806b1744d9d93: Status 404 returned error can't find the container with id 0299f70a1dc486c61a3fe6fb3c8f9fa35f50cd736e9734056b3806b1744d9d93 Jul 22 02:29:09.825289 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:09.825199 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cb8d48668-9mpmn"] Jul 22 02:29:09.830548 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:29:09.830405 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7bd1a08_d4ef_4f8d_9b6c_baa25b8af3ab.slice/crio-ee37322f94676a0f69eef5fcc9e8e5090dd0e90d44207c97b8d3d753ec52fb1f WatchSource:0}: Error finding container ee37322f94676a0f69eef5fcc9e8e5090dd0e90d44207c97b8d3d753ec52fb1f: Status 404 returned error can't find the container with id ee37322f94676a0f69eef5fcc9e8e5090dd0e90d44207c97b8d3d753ec52fb1f Jul 22 02:29:09.869063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:09.869030 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:09.970641 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:09.970609 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:10.052374 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.052327 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:10.052541 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.052384 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7nf8q"] Jul 22 02:29:10.074311 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.074130 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7nf8q"] Jul 22 02:29:10.074752 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.074710 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7nf8q" Jul 22 02:29:10.077391 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.077368 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 22 02:29:10.077391 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.077388 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-2tcfc\"" Jul 22 02:29:10.099998 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.099939 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-phhmt" event={"ID":"2500f8f3-355d-47ef-af40-da1b1e9a6ccd","Type":"ContainerStarted","Data":"6d3862b90120f11248906a1cae8f04d313197ab9b682642fe1596c3e1f4a900c"} Jul 22 02:29:10.101067 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.101040 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57c45b69d8-f5hj6" event={"ID":"f13956b7-8c9e-4a56-8bd9-8b7acf0af176","Type":"ContainerStarted","Data":"0299f70a1dc486c61a3fe6fb3c8f9fa35f50cd736e9734056b3806b1744d9d93"} Jul 22 02:29:10.102358 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.102325 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d7b6cf945-r86gn" event={"ID":"b6f86fa5-28b5-4a34-a437-28756a33fdf4","Type":"ContainerStarted","Data":"20e3b3bc40dc171abe9c538c521703c9ead865061bc740d7d6b0ff45357c3244"} Jul 22 02:29:10.103788 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.103763 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cjr5p" event={"ID":"72d84003-0a34-4663-be5f-09d9d7a14498","Type":"ContainerStarted","Data":"4a1cf26f2f2482b472695514098e3ace0a0759415924ff93cff41343b5c1fbe1"} Jul 22 02:29:10.105038 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.105017 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" event={"ID":"145b0768-765c-48a5-85f6-c55397a4fe0d","Type":"ContainerStarted","Data":"5ed10b63146fa9310b09342e4fd549f6784dd3638fc9555f7c1b7f1d483cc222"} Jul 22 02:29:10.106223 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.106203 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq" event={"ID":"d68cb4ff-40dc-4020-bac3-0db866ecf648","Type":"ContainerStarted","Data":"47724a836c0a5e0b7ddbf6b1a8e3c45c0a50cb2036472856bc5af9324ceafae3"} Jul 22 02:29:10.107857 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.107836 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" event={"ID":"9f11a91e-5064-4838-bd58-7ab3369cc834","Type":"ContainerStarted","Data":"bbdc0b7387ef89115bd662cfa517c4fbba2f5dacf549f43837da43789959ec0d"} Jul 22 02:29:10.109672 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.109652 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8dnzm" event={"ID":"7f7165bc-e726-4334-8e57-49c05f994762","Type":"ContainerStarted","Data":"e36cc9ff8a28ae34cb6bac6b07911a54e1cb1ab67e6ecc7a796c07c0e3b942b0"} Jul 22 02:29:10.111445 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.111422 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6gfmw" event={"ID":"7243e221-39df-4cbe-bd32-302e9744977d","Type":"ContainerStarted","Data":"af403af88f6d6df3e06c2075fd37bfec82b622d818f7066875f8a76c7345922f"} Jul 22 02:29:10.111537 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.111468 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6gfmw" event={"ID":"7243e221-39df-4cbe-bd32-302e9744977d","Type":"ContainerStarted","Data":"47ae33d27fc1a0bdd08ed0bb3894bf6b3c43ce9b142cd8471448d29bc5444121"} Jul 22 02:29:10.111537 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.111509 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6gfmw" Jul 22 02:29:10.112779 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.112756 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb8d48668-9mpmn" event={"ID":"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab","Type":"ContainerStarted","Data":"ee37322f94676a0f69eef5fcc9e8e5090dd0e90d44207c97b8d3d753ec52fb1f"} Jul 22 02:29:10.114271 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.114251 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-66nws" event={"ID":"f1da4aa5-c459-4bfd-9a72-5e179d8af2e3","Type":"ContainerStarted","Data":"bf00c34355e574a4d8dcb42c9105d0f9523afcd40549035e08909896540ff1cd"} Jul 22 02:29:10.121316 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.121267 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cjr5p" podStartSLOduration=66.61925579 podStartE2EDuration="1m14.121253774s" totalImagesPullingTime="7.501997984s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:27:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:10.120441461 +0000 UTC m=+74.969298292" watchObservedRunningTime="2026-07-22 02:29:10.121253774 +0000 UTC m=+74.970110584" Jul 22 02:29:10.140408 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.140350 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-85lxl" podStartSLOduration=61.519256178 podStartE2EDuration="1m8.140330042s" totalImagesPullingTime="6.621073864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:10.139348711 +0000 UTC m=+74.988205543" watchObservedRunningTime="2026-07-22 02:29:10.140330042 +0000 UTC m=+74.989186877" Jul 22 02:29:10.156354 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.156307 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-66nws" podStartSLOduration=33.646029857 podStartE2EDuration="40.156290906s" totalImagesPullingTime="6.510261049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:10.155159344 +0000 UTC m=+75.004016177" watchObservedRunningTime="2026-07-22 02:29:10.156290906 +0000 UTC m=+75.005147801" Jul 22 02:29:10.175807 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.175773 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fddc14b7-ef5f-497b-af06-a15a0e952656-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-7nf8q\" (UID: \"fddc14b7-ef5f-497b-af06-a15a0e952656\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7nf8q" Jul 22 02:29:10.179086 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.179039 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6gfmw" podStartSLOduration=34.653982306 podStartE2EDuration="41.179022368s" totalImagesPullingTime="6.525040062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:10.178212777 +0000 UTC m=+75.027069610" watchObservedRunningTime="2026-07-22 02:29:10.179022368 +0000 UTC m=+75.027879178" Jul 22 02:29:10.277198 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.277161 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fddc14b7-ef5f-497b-af06-a15a0e952656-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-7nf8q\" (UID: \"fddc14b7-ef5f-497b-af06-a15a0e952656\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7nf8q" Jul 22 02:29:10.279638 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.279613 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fddc14b7-ef5f-497b-af06-a15a0e952656-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-7nf8q\" (UID: \"fddc14b7-ef5f-497b-af06-a15a0e952656\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7nf8q" Jul 22 02:29:10.321722 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.321693 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:10.386169 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.386136 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7nf8q" Jul 22 02:29:10.414631 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.414596 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:10.515389 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.515229 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:10.568598 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.568565 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7nf8q"] Jul 22 02:29:10.575532 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:29:10.574222 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfddc14b7_ef5f_497b_af06_a15a0e952656.slice/crio-28572346a56a6a64299a7aef9a560810229427178755ea704479d1a436088690 WatchSource:0}: Error finding container 28572346a56a6a64299a7aef9a560810229427178755ea704479d1a436088690: Status 404 returned error can't find the container with id 28572346a56a6a64299a7aef9a560810229427178755ea704479d1a436088690 Jul 22 02:29:10.914475 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.914332 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:10.989111 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:10.989053 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:11.126588 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:11.125501 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7nf8q" event={"ID":"fddc14b7-ef5f-497b-af06-a15a0e952656","Type":"ContainerStarted","Data":"28572346a56a6a64299a7aef9a560810229427178755ea704479d1a436088690"} Jul 22 02:29:11.573237 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:11.573186 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:11.668445 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:11.668388 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:11.918747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:11.918666 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:12.020874 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:12.020823 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:12.092636 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:12.092567 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:19.702305 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:19.702261 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:19.781713 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:19.781659 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:20.128546 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:20.128516 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6gfmw" Jul 22 02:29:21.198436 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:21.198040 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8dnzm" event={"ID":"7f7165bc-e726-4334-8e57-49c05f994762","Type":"ContainerStarted","Data":"1c8122e1042ff7f537e08cc26367961eec39824bc848c5615ae7eb7af3543d8d"} Jul 22 02:29:21.203925 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:21.203837 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb8d48668-9mpmn" event={"ID":"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab","Type":"ContainerStarted","Data":"e68eee3ec0c5ad6ada3c3b1010ae683dfd3edf0a695fba962d8cbe3ad456ff1b"} Jul 22 02:29:21.205742 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:21.205718 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7nf8q" event={"ID":"fddc14b7-ef5f-497b-af06-a15a0e952656","Type":"ContainerStarted","Data":"1d2800aacd436f984b4866b0bdb3af860b6e103e25577793ccb65090f6aa9272"} Jul 22 02:29:21.206285 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:21.206250 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7nf8q" Jul 22 02:29:21.212664 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:21.212432 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7nf8q" Jul 22 02:29:21.221691 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:21.221388 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8dnzm" podStartSLOduration=16.662080361 podStartE2EDuration="41.221365716s" totalImagesPullingTime="24.559285355s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:28:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:21.218403461 +0000 UTC m=+86.067260294" watchObservedRunningTime="2026-07-22 02:29:21.221365716 +0000 UTC m=+86.070222550" Jul 22 02:29:21.243785 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:21.243716 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7nf8q" podStartSLOduration=0.838133504 podStartE2EDuration="11.243694259s" totalImagesPullingTime="10.405560755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:29:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:21.242126648 +0000 UTC m=+86.090983501" watchObservedRunningTime="2026-07-22 02:29:21.243694259 +0000 UTC m=+86.092551092" Jul 22 02:29:21.267969 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:21.266958 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cb8d48668-9mpmn" podStartSLOduration=5.148862197 podStartE2EDuration="16.266920239s" totalImagesPullingTime="11.118058042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:29:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:21.265668966 +0000 UTC m=+86.114525799" watchObservedRunningTime="2026-07-22 02:29:21.266920239 +0000 UTC m=+86.115777051" Jul 22 02:29:21.477904 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:21.477866 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:21.585135 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:21.585106 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:21.652193 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:21.652162 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:22.075567 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:22.075533 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:22.127929 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:22.127873 2579 patch_prober.go:28] interesting pod/image-registry-fd9dff84f-67px6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 22 02:29:22.128115 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:22.127958 2579 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-fd9dff84f-67px6" podUID="266db83d-8440-47b6-a7c7-8fb294a00554" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 02:29:22.177088 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:22.177054 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:22.218868 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:22.218829 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" event={"ID":"145b0768-765c-48a5-85f6-c55397a4fe0d","Type":"ContainerStarted","Data":"807ca674f44141e31817baa549170cc80e0a53bc62b8a34014423492706adb10"} Jul 22 02:29:22.218868 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:22.218883 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" event={"ID":"145b0768-765c-48a5-85f6-c55397a4fe0d","Type":"ContainerStarted","Data":"1d56adae1b12d9eb64c005c6f43d1155126141b43d9f9954fd5c11fd25dc3ec8"} Jul 22 02:29:22.219400 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:22.218901 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" event={"ID":"145b0768-765c-48a5-85f6-c55397a4fe0d","Type":"ContainerStarted","Data":"3b209684857ac250af71b8a1e7b536699faf37bcb351e058ac0f82b15293d662"} Jul 22 02:29:22.221112 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:22.221080 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq" event={"ID":"d68cb4ff-40dc-4020-bac3-0db866ecf648","Type":"ContainerStarted","Data":"7dab75416c44c350e2037e731a0fd247ec919b4e232d23581f85a6fed7754d80"} Jul 22 02:29:22.221256 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:22.221173 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq" Jul 22 02:29:22.223428 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:22.223400 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57c45b69d8-f5hj6" event={"ID":"f13956b7-8c9e-4a56-8bd9-8b7acf0af176","Type":"ContainerStarted","Data":"edd206b92db9bf5bc6445afed13c9da61ba74277afcebfe753bea46820ce8949"} Jul 22 02:29:22.223631 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:22.223612 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq" Jul 22 02:29:22.225958 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:22.225917 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d7b6cf945-r86gn" event={"ID":"b6f86fa5-28b5-4a34-a437-28756a33fdf4","Type":"ContainerStarted","Data":"3bd528218ca9bcbc67269ac59d246d2f224848f3c955d3e31e20228dddfcd9f1"} Jul 22 02:29:22.240901 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:22.240842 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7875bd8555-56v2g" podStartSLOduration=5.959510368 podStartE2EDuration="17.24082236s" totalImagesPullingTime="11.281311992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:29:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:22.239047885 +0000 UTC m=+87.087904719" watchObservedRunningTime="2026-07-22 02:29:22.24082236 +0000 UTC m=+87.089679193" Jul 22 02:29:22.261394 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:22.261336 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d7b6cf945-r86gn" podStartSLOduration=4.981258622 podStartE2EDuration="16.261318487s" totalImagesPullingTime="11.280059865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:22.259620139 +0000 UTC m=+87.108476968" watchObservedRunningTime="2026-07-22 02:29:22.261318487 +0000 UTC m=+87.110175299" Jul 22 02:29:22.291773 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:22.291726 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-749bd4fb67-bzfxq" podStartSLOduration=5.992066288 podStartE2EDuration="17.291708087s" totalImagesPullingTime="11.299641799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:29:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:22.275105866 +0000 UTC m=+87.123962699" watchObservedRunningTime="2026-07-22 02:29:22.291708087 +0000 UTC m=+87.140564919" Jul 22 02:29:22.618424 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:22.618377 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:23.406696 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:23.406644 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:23.507157 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:23.507120 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:25.078184 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:25.078156 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:29:25.100511 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:25.100455 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57c45b69d8-f5hj6" podStartSLOduration=8.584881646 podStartE2EDuration="20.100436211s" totalImagesPullingTime="11.515554565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:29:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:22.291425211 +0000 UTC m=+87.140282045" watchObservedRunningTime="2026-07-22 02:29:25.100436211 +0000 UTC m=+89.949293043" Jul 22 02:29:25.205852 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:25.205809 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:25.275687 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:25.275648 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:25.342703 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:25.342619 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:25.442698 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:25.442664 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:25.654588 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:25.654505 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:26.221025 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:26.220986 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:26.221512 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:26.221105 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:26.227013 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:26.226987 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:26.250888 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:26.250858 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:26.393806 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:26.393750 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:27.340107 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:27.340067 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:27.340541 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:27.340146 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:27.345561 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:27.345536 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:28.220433 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.220364 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:28.259905 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.259874 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:29:28.318100 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.317980 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cb8d48668-9mpmn"] Jul 22 02:29:28.450291 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.450259 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:28.542206 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.541400 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2s67d"] Jul 22 02:29:28.564813 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.564700 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:28.629725 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.629682 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:28.717968 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.717533 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.721795 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.720996 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 02:29:28.721795 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.721244 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-z2bb6\"" Jul 22 02:29:28.721795 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.721629 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 02:29:28.723636 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.722327 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 02:29:28.726094 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.721423 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 02:29:28.731989 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.731930 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:28.830895 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.830844 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:28.861666 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.861609 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-accelerators-collector-config\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.861666 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.861669 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-root\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.861923 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.861722 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-wtmp\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.861923 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.861747 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-sys\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.861923 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.861832 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gx6f\" (UniqueName: \"kubernetes.io/projected/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-kube-api-access-7gx6f\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.861923 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.861863 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-tls\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.861923 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.861902 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-metrics-client-ca\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.862184 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.861958 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-textfile\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.862184 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.862010 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.963014 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.962925 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-wtmp\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.963721 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.963416 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-sys\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.963721 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.963120 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-wtmp\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.963721 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.963470 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gx6f\" (UniqueName: \"kubernetes.io/projected/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-kube-api-access-7gx6f\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.963721 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.963524 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-tls\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.963721 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.963568 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-metrics-client-ca\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.963721 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.963607 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-textfile\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.963721 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.963660 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.963721 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.963691 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-accelerators-collector-config\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.963721 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.963716 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-sys\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.964240 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.963728 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-root\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.964240 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:29:28.964064 2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 22 02:29:28.964240 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:29:28.964129 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-tls podName:b92cff58-42df-4c45-88f4-b3e5bc97f9aa nodeName:}" failed. No retries permitted until 2026-07-22 02:29:29.464106768 +0000 UTC m=+94.312963582 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-tls") pod "node-exporter-2s67d" (UID: "b92cff58-42df-4c45-88f4-b3e5bc97f9aa") : secret "node-exporter-tls" not found Jul 22 02:29:28.964240 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.964218 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-root\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.964444 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.964243 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-textfile\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.965187 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.964721 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-metrics-client-ca\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.965187 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.965123 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-accelerators-collector-config\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.967771 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.967752 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:28.976164 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:28.976135 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gx6f\" (UniqueName: \"kubernetes.io/projected/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-kube-api-access-7gx6f\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:29.083992 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:29.083835 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:29.184317 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:29.184279 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:29.285649 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:29.285615 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:29.388647 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:29.388563 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:29.469325 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:29.469289 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-tls\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:29.472366 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:29.472337 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b92cff58-42df-4c45-88f4-b3e5bc97f9aa-node-exporter-tls\") pod \"node-exporter-2s67d\" (UID: \"b92cff58-42df-4c45-88f4-b3e5bc97f9aa\") " pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:29.489422 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:29.489369 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:29.592192 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:29.592151 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:29.633829 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:29.633798 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2s67d" Jul 22 02:29:29.903044 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:29.903006 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:30.197277 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:30.197242 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:30.256927 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:30.256891 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:30.428935 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:30.428904 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:30.526293 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:30.526216 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:30.713496 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:30.713411 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:30.790078 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:30.789984 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:31.011107 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:29:31.011082 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb92cff58_42df_4c45_88f4_b3e5bc97f9aa.slice/crio-a2be0ead2598020a1452308227fb2fbd214dcc94c593e17105f0c0d6f05bdf87 WatchSource:0}: Error finding container a2be0ead2598020a1452308227fb2fbd214dcc94c593e17105f0c0d6f05bdf87: Status 404 returned error can't find the container with id a2be0ead2598020a1452308227fb2fbd214dcc94c593e17105f0c0d6f05bdf87 Jul 22 02:29:31.271713 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:31.271675 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2s67d" event={"ID":"b92cff58-42df-4c45-88f4-b3e5bc97f9aa","Type":"ContainerStarted","Data":"a2be0ead2598020a1452308227fb2fbd214dcc94c593e17105f0c0d6f05bdf87"} Jul 22 02:29:31.442517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:31.442468 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:31.547352 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:31.547273 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:31.795530 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:31.795502 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:31.886598 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:31.886548 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:32.279144 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:32.279060 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-phhmt" event={"ID":"2500f8f3-355d-47ef-af40-da1b1e9a6ccd","Type":"ContainerStarted","Data":"bca0709acb2f34806d838bf3fbe5b71171662f2f22c1dd27f343c2712dfedb3e"} Jul 22 02:29:32.279325 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:32.279308 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-phhmt" Jul 22 02:29:32.296588 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:32.296559 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-phhmt" Jul 22 02:29:32.300361 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:32.300305 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-phhmt" podStartSLOduration=5.651315956 podStartE2EDuration="27.300287111s" totalImagesPullingTime="21.648971155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:29:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:32.298999553 +0000 UTC m=+97.147856385" watchObservedRunningTime="2026-07-22 02:29:32.300287111 +0000 UTC m=+97.149143944" Jul 22 02:29:33.059472 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:33.059433 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9kvjj" Jul 22 02:29:33.250367 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:33.250323 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:33.295025 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:33.294986 2579 generic.go:356] "Generic (PLEG): container finished" podID="b92cff58-42df-4c45-88f4-b3e5bc97f9aa" containerID="ede95d4ac0ce738ab3eef1c2a7d5e8faa7873e106c890dae7e14b511f9be9a0c" exitCode=0 Jul 22 02:29:33.295214 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:33.295072 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2s67d" event={"ID":"b92cff58-42df-4c45-88f4-b3e5bc97f9aa","Type":"ContainerDied","Data":"ede95d4ac0ce738ab3eef1c2a7d5e8faa7873e106c890dae7e14b511f9be9a0c"} Jul 22 02:29:33.343910 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:33.343863 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:33.629812 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:33.629723 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:33.737030 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:33.736988 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:34.126517 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:34.126482 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:34.228039 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:34.227996 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:34.302497 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:34.302458 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2s67d" event={"ID":"b92cff58-42df-4c45-88f4-b3e5bc97f9aa","Type":"ContainerStarted","Data":"bde3b1dd983b39be925e55df1c0aa165a65f1f8dc4c417b369ad5cd183feed09"} Jul 22 02:29:34.302722 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:34.302523 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2s67d" event={"ID":"b92cff58-42df-4c45-88f4-b3e5bc97f9aa","Type":"ContainerStarted","Data":"7bd323d638399712bc13cd7868c9fe27295ecdc39c5dc2f4c599983487da5725"} Jul 22 02:29:34.326569 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:34.326512 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2s67d" podStartSLOduration=4.741867748 podStartE2EDuration="6.326495025s" totalImagesPullingTime="1.474623941s" totalInitContainerRuntime="110.003336ms" isStatefulPod=true podCreationTimestamp="2026-07-22 02:29:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:34.32458668 +0000 UTC m=+99.173443526" watchObservedRunningTime="2026-07-22 02:29:34.326495025 +0000 UTC m=+99.175351856" Jul 22 02:29:34.649387 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:34.649353 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-fd9dff84f-67px6"] Jul 22 02:29:34.746473 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:34.746423 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:34.912913 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:34.912829 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:35.009092 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:35.009056 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:37.841214 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:37.841180 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:37.942664 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:37.942624 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:38.042347 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:38.042302 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:38.397277 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:38.397243 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:38.496439 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:38.496406 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:38.812762 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:38.812671 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:38.914380 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:38.914330 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:39.336206 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:39.336171 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:39.410186 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:39.410149 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:40.885519 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:40.885490 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:40.988378 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:40.988343 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:42.570529 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.570496 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65dffcd474-g2cnz"] Jul 22 02:29:42.674295 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.674254 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65dffcd474-g2cnz"] Jul 22 02:29:42.674472 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.674366 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.795020 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.794932 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-oauth-config\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.795207 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.795057 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-service-ca\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.795207 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.795087 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-trusted-ca-bundle\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.795207 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.795118 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-serving-cert\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.795207 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.795140 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-oauth-serving-cert\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.795364 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.795199 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-config\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.795364 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.795244 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tdfg\" (UniqueName: \"kubernetes.io/projected/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-kube-api-access-2tdfg\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.896128 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.896089 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-oauth-config\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.896340 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.896144 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-service-ca\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.896340 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.896164 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-trusted-ca-bundle\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.896340 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.896197 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-serving-cert\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.896340 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.896217 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-oauth-serving-cert\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.896340 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.896245 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-config\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.896579 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.896380 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tdfg\" (UniqueName: \"kubernetes.io/projected/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-kube-api-access-2tdfg\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.897173 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.897145 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-oauth-serving-cert\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.897330 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.897177 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-service-ca\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.897330 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.897297 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-trusted-ca-bundle\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.897715 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.897696 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-config\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.898969 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.898928 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-serving-cert\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.899074 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.899028 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-oauth-config\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.904882 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.904856 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tdfg\" (UniqueName: \"kubernetes.io/projected/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-kube-api-access-2tdfg\") pod \"console-65dffcd474-g2cnz\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:42.983352 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:42.983317 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:43.150131 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:43.150099 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65dffcd474-g2cnz"] Jul 22 02:29:43.152600 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:29:43.152568 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ffd79fc_2d6f_4892_a913_8ba8116c8fc9.slice/crio-05537e7b48d42435c682583daf7416b7e3103d6f583ce3d143ce13e0de45e52b WatchSource:0}: Error finding container 05537e7b48d42435c682583daf7416b7e3103d6f583ce3d143ce13e0de45e52b: Status 404 returned error can't find the container with id 05537e7b48d42435c682583daf7416b7e3103d6f583ce3d143ce13e0de45e52b Jul 22 02:29:43.345888 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:43.345851 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65dffcd474-g2cnz" event={"ID":"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9","Type":"ContainerStarted","Data":"87d77a447e6f37ad7e2aa65d6723c628013f1dd41a294c21cfa634495b3f2659"} Jul 22 02:29:43.346095 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:43.345905 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65dffcd474-g2cnz" event={"ID":"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9","Type":"ContainerStarted","Data":"05537e7b48d42435c682583daf7416b7e3103d6f583ce3d143ce13e0de45e52b"} Jul 22 02:29:43.364026 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:43.363976 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65dffcd474-g2cnz" podStartSLOduration=1.363889066 podStartE2EDuration="1.363889066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:29:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:43.363824234 +0000 UTC m=+108.212681067" watchObservedRunningTime="2026-07-22 02:29:43.363889066 +0000 UTC m=+108.212745898" Jul 22 02:29:44.352097 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:44.352064 2579 generic.go:356] "Generic (PLEG): container finished" podID="3bf725ad-6951-48c4-b2e4-0d898eec59ed" containerID="c72a2446f8295a32251012d4fb877b64497a32a2b97bc57817cfab230c955191" exitCode=0 Jul 22 02:29:44.352564 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:44.352143 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv" event={"ID":"3bf725ad-6951-48c4-b2e4-0d898eec59ed","Type":"ContainerDied","Data":"c72a2446f8295a32251012d4fb877b64497a32a2b97bc57817cfab230c955191"} Jul 22 02:29:44.352564 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:44.352553 2579 scope.go:117] "RemoveContainer" containerID="c72a2446f8295a32251012d4fb877b64497a32a2b97bc57817cfab230c955191" Jul 22 02:29:44.353606 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:44.353586 2579 generic.go:356] "Generic (PLEG): container finished" podID="1239da74-1398-4bd0-9d28-98bacba6839a" containerID="836b0973f4f173e1a7ed437374b9a07b7b88d45957f7a5f870beddf3ba0ff1a3" exitCode=0 Jul 22 02:29:44.353700 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:44.353659 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf" event={"ID":"1239da74-1398-4bd0-9d28-98bacba6839a","Type":"ContainerDied","Data":"836b0973f4f173e1a7ed437374b9a07b7b88d45957f7a5f870beddf3ba0ff1a3"} Jul 22 02:29:44.353919 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:44.353904 2579 scope.go:117] "RemoveContainer" containerID="836b0973f4f173e1a7ed437374b9a07b7b88d45957f7a5f870beddf3ba0ff1a3" Jul 22 02:29:45.359721 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:45.359686 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-924jv" event={"ID":"3bf725ad-6951-48c4-b2e4-0d898eec59ed","Type":"ContainerStarted","Data":"19974634fb99c20fc9d92be7c4c15189c6ecccd1e6ad0230a4a48db0002e8660"} Jul 22 02:29:45.361165 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:45.361144 2579 generic.go:356] "Generic (PLEG): container finished" podID="858fe6a9-2e1c-48c7-8e67-12119323253e" containerID="c6e6100d12767c56eb328a7271348ef3b5c857f4511a37efff243ca61dc19255" exitCode=0 Jul 22 02:29:45.361287 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:45.361216 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-tq2tv" event={"ID":"858fe6a9-2e1c-48c7-8e67-12119323253e","Type":"ContainerDied","Data":"c6e6100d12767c56eb328a7271348ef3b5c857f4511a37efff243ca61dc19255"} Jul 22 02:29:45.361567 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:45.361549 2579 scope.go:117] "RemoveContainer" containerID="c6e6100d12767c56eb328a7271348ef3b5c857f4511a37efff243ca61dc19255" Jul 22 02:29:45.363175 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:45.363153 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wxxgf" event={"ID":"1239da74-1398-4bd0-9d28-98bacba6839a","Type":"ContainerStarted","Data":"b1acf53ec51be1a751c4fbd56415690a1af0368e4b6b0d1fad2e75f01a2053ec"} Jul 22 02:29:46.370435 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:46.370401 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-tq2tv" event={"ID":"858fe6a9-2e1c-48c7-8e67-12119323253e","Type":"ContainerStarted","Data":"b9ac80cb4c0e269fede56e599883fe1025c3704efc2548009c9739b64fda7873"} Jul 22 02:29:47.773812 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:47.773780 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:47.874722 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:47.874675 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:52.983797 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:52.983755 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:52.984347 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:52.983831 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:52.988486 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:52.988461 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:53.345100 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.345039 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5cb8d48668-9mpmn" podUID="f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab" containerName="console" containerID="cri-o://e68eee3ec0c5ad6ada3c3b1010ae683dfd3edf0a695fba962d8cbe3ad456ff1b" gracePeriod=15 Jul 22 02:29:53.399502 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.399467 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:53.405569 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.405546 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:29:53.459138 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.459104 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d7b6cf945-r86gn"] Jul 22 02:29:53.491846 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.491799 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:53.588879 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.588844 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:53.648698 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.648668 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:53.653175 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.653152 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:53.710789 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.710754 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:29:53.790920 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.790880 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-config\") pod \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " Jul 22 02:29:53.790920 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.790923 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-oauth-config\") pod \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " Jul 22 02:29:53.791202 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.790998 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-serving-cert\") pod \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " Jul 22 02:29:53.791202 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.791019 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-oauth-serving-cert\") pod \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " Jul 22 02:29:53.791202 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.791175 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-service-ca\") pod \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " Jul 22 02:29:53.791356 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.791265 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrv7p\" (UniqueName: \"kubernetes.io/projected/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-kube-api-access-hrv7p\") pod \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\" (UID: \"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab\") " Jul 22 02:29:53.791525 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.791414 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-config" (OuterVolumeSpecName: "console-config") pod "f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab" (UID: "f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:29:53.791596 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.791579 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-config\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:29:53.791700 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.791582 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-service-ca" (OuterVolumeSpecName: "service-ca") pod "f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab" (UID: "f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:29:53.791919 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.791779 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab" (UID: "f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:29:53.793595 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.793565 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab" (UID: "f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:29:53.793692 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.793621 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-kube-api-access-hrv7p" (OuterVolumeSpecName: "kube-api-access-hrv7p") pod "f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab" (UID: "f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab"). InnerVolumeSpecName "kube-api-access-hrv7p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:29:53.793692 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.793633 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab" (UID: "f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:29:53.892382 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.892301 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-service-ca\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:29:53.892382 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.892327 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrv7p\" (UniqueName: \"kubernetes.io/projected/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-kube-api-access-hrv7p\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:29:53.892382 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.892338 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-oauth-config\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:29:53.892382 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.892347 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-console-serving-cert\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:29:53.892382 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:53.892355 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab-oauth-serving-cert\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:29:54.411660 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:54.411621 2579 generic.go:356] "Generic (PLEG): container finished" podID="f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab" containerID="e68eee3ec0c5ad6ada3c3b1010ae683dfd3edf0a695fba962d8cbe3ad456ff1b" exitCode=0 Jul 22 02:29:54.412251 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:54.411684 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb8d48668-9mpmn" Jul 22 02:29:54.412251 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:54.411708 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb8d48668-9mpmn" event={"ID":"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab","Type":"ContainerDied","Data":"e68eee3ec0c5ad6ada3c3b1010ae683dfd3edf0a695fba962d8cbe3ad456ff1b"} Jul 22 02:29:54.412251 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:54.411784 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb8d48668-9mpmn" event={"ID":"f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab","Type":"ContainerDied","Data":"ee37322f94676a0f69eef5fcc9e8e5090dd0e90d44207c97b8d3d753ec52fb1f"} Jul 22 02:29:54.412251 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:54.411805 2579 scope.go:117] "RemoveContainer" containerID="e68eee3ec0c5ad6ada3c3b1010ae683dfd3edf0a695fba962d8cbe3ad456ff1b" Jul 22 02:29:54.420642 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:54.420623 2579 scope.go:117] "RemoveContainer" containerID="e68eee3ec0c5ad6ada3c3b1010ae683dfd3edf0a695fba962d8cbe3ad456ff1b" Jul 22 02:29:54.420966 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:29:54.420919 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e68eee3ec0c5ad6ada3c3b1010ae683dfd3edf0a695fba962d8cbe3ad456ff1b\": container with ID starting with e68eee3ec0c5ad6ada3c3b1010ae683dfd3edf0a695fba962d8cbe3ad456ff1b not found: ID does not exist" containerID="e68eee3ec0c5ad6ada3c3b1010ae683dfd3edf0a695fba962d8cbe3ad456ff1b" Jul 22 02:29:54.421065 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:54.421031 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e68eee3ec0c5ad6ada3c3b1010ae683dfd3edf0a695fba962d8cbe3ad456ff1b"} err="failed to get container status \"e68eee3ec0c5ad6ada3c3b1010ae683dfd3edf0a695fba962d8cbe3ad456ff1b\": rpc error: code = NotFound desc = could not find container \"e68eee3ec0c5ad6ada3c3b1010ae683dfd3edf0a695fba962d8cbe3ad456ff1b\": container with ID starting with e68eee3ec0c5ad6ada3c3b1010ae683dfd3edf0a695fba962d8cbe3ad456ff1b not found: ID does not exist" Jul 22 02:29:54.432720 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:54.432684 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cb8d48668-9mpmn"] Jul 22 02:29:54.436541 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:54.436514 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cb8d48668-9mpmn"] Jul 22 02:29:55.626924 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:55.626889 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab" path="/var/lib/kubelet/pods/f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab/volumes" Jul 22 02:29:59.674763 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:59.674702 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-fd9dff84f-67px6" podUID="266db83d-8440-47b6-a7c7-8fb294a00554" containerName="registry" containerID="cri-o://49720997cc38ded0e37fc10c1384868f149976b2084f9cdc7261b8355347e588" gracePeriod=30 Jul 22 02:29:59.959124 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:29:59.959099 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:30:00.050357 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.050316 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls\") pod \"266db83d-8440-47b6-a7c7-8fb294a00554\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " Jul 22 02:30:00.050514 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.050372 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/266db83d-8440-47b6-a7c7-8fb294a00554-trusted-ca\") pod \"266db83d-8440-47b6-a7c7-8fb294a00554\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " Jul 22 02:30:00.050514 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.050401 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/266db83d-8440-47b6-a7c7-8fb294a00554-registry-certificates\") pod \"266db83d-8440-47b6-a7c7-8fb294a00554\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " Jul 22 02:30:00.050514 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.050424 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/266db83d-8440-47b6-a7c7-8fb294a00554-installation-pull-secrets\") pod \"266db83d-8440-47b6-a7c7-8fb294a00554\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " Jul 22 02:30:00.050514 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.050461 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/266db83d-8440-47b6-a7c7-8fb294a00554-image-registry-private-configuration\") pod \"266db83d-8440-47b6-a7c7-8fb294a00554\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " Jul 22 02:30:00.050709 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.050557 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-bound-sa-token\") pod \"266db83d-8440-47b6-a7c7-8fb294a00554\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " Jul 22 02:30:00.050709 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.050657 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fs9zd\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-kube-api-access-fs9zd\") pod \"266db83d-8440-47b6-a7c7-8fb294a00554\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " Jul 22 02:30:00.050709 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.050703 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/266db83d-8440-47b6-a7c7-8fb294a00554-ca-trust-extracted\") pod \"266db83d-8440-47b6-a7c7-8fb294a00554\" (UID: \"266db83d-8440-47b6-a7c7-8fb294a00554\") " Jul 22 02:30:00.050840 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.050781 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/266db83d-8440-47b6-a7c7-8fb294a00554-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "266db83d-8440-47b6-a7c7-8fb294a00554" (UID: "266db83d-8440-47b6-a7c7-8fb294a00554"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:30:00.050895 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.050841 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/266db83d-8440-47b6-a7c7-8fb294a00554-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "266db83d-8440-47b6-a7c7-8fb294a00554" (UID: "266db83d-8440-47b6-a7c7-8fb294a00554"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:30:00.051489 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.051037 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/266db83d-8440-47b6-a7c7-8fb294a00554-trusted-ca\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:30:00.051489 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.051063 2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/266db83d-8440-47b6-a7c7-8fb294a00554-registry-certificates\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:30:00.053605 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.053551 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "266db83d-8440-47b6-a7c7-8fb294a00554" (UID: "266db83d-8440-47b6-a7c7-8fb294a00554"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:30:00.053730 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.053645 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/266db83d-8440-47b6-a7c7-8fb294a00554-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "266db83d-8440-47b6-a7c7-8fb294a00554" (UID: "266db83d-8440-47b6-a7c7-8fb294a00554"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:30:00.053786 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.053752 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "266db83d-8440-47b6-a7c7-8fb294a00554" (UID: "266db83d-8440-47b6-a7c7-8fb294a00554"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:30:00.053933 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.053906 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-kube-api-access-fs9zd" (OuterVolumeSpecName: "kube-api-access-fs9zd") pod "266db83d-8440-47b6-a7c7-8fb294a00554" (UID: "266db83d-8440-47b6-a7c7-8fb294a00554"). InnerVolumeSpecName "kube-api-access-fs9zd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:30:00.054040 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.054006 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/266db83d-8440-47b6-a7c7-8fb294a00554-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "266db83d-8440-47b6-a7c7-8fb294a00554" (UID: "266db83d-8440-47b6-a7c7-8fb294a00554"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:30:00.062117 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.062085 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/266db83d-8440-47b6-a7c7-8fb294a00554-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "266db83d-8440-47b6-a7c7-8fb294a00554" (UID: "266db83d-8440-47b6-a7c7-8fb294a00554"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:30:00.151998 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.151938 2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/266db83d-8440-47b6-a7c7-8fb294a00554-image-registry-private-configuration\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:30:00.151998 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.151999 2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-bound-sa-token\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:30:00.152210 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.152014 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fs9zd\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-kube-api-access-fs9zd\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:30:00.152210 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.152026 2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/266db83d-8440-47b6-a7c7-8fb294a00554-ca-trust-extracted\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:30:00.152210 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.152037 2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/266db83d-8440-47b6-a7c7-8fb294a00554-registry-tls\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:30:00.152210 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.152048 2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/266db83d-8440-47b6-a7c7-8fb294a00554-installation-pull-secrets\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:30:00.445928 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.445895 2579 generic.go:356] "Generic (PLEG): container finished" podID="266db83d-8440-47b6-a7c7-8fb294a00554" containerID="49720997cc38ded0e37fc10c1384868f149976b2084f9cdc7261b8355347e588" exitCode=0 Jul 22 02:30:00.446116 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.445965 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fd9dff84f-67px6" Jul 22 02:30:00.446116 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.445979 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fd9dff84f-67px6" event={"ID":"266db83d-8440-47b6-a7c7-8fb294a00554","Type":"ContainerDied","Data":"49720997cc38ded0e37fc10c1384868f149976b2084f9cdc7261b8355347e588"} Jul 22 02:30:00.446116 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.446037 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fd9dff84f-67px6" event={"ID":"266db83d-8440-47b6-a7c7-8fb294a00554","Type":"ContainerDied","Data":"a06429bed930a5e2dd9372a7cd463eb84f1e5afc06f04942c61e575210661b3e"} Jul 22 02:30:00.446116 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.446057 2579 scope.go:117] "RemoveContainer" containerID="49720997cc38ded0e37fc10c1384868f149976b2084f9cdc7261b8355347e588" Jul 22 02:30:00.467619 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.466121 2579 scope.go:117] "RemoveContainer" containerID="49720997cc38ded0e37fc10c1384868f149976b2084f9cdc7261b8355347e588" Jul 22 02:30:00.467911 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:30:00.467876 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49720997cc38ded0e37fc10c1384868f149976b2084f9cdc7261b8355347e588\": container with ID starting with 49720997cc38ded0e37fc10c1384868f149976b2084f9cdc7261b8355347e588 not found: ID does not exist" containerID="49720997cc38ded0e37fc10c1384868f149976b2084f9cdc7261b8355347e588" Jul 22 02:30:00.468020 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.467929 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49720997cc38ded0e37fc10c1384868f149976b2084f9cdc7261b8355347e588"} err="failed to get container status \"49720997cc38ded0e37fc10c1384868f149976b2084f9cdc7261b8355347e588\": rpc error: code = NotFound desc = could not find container \"49720997cc38ded0e37fc10c1384868f149976b2084f9cdc7261b8355347e588\": container with ID starting with 49720997cc38ded0e37fc10c1384868f149976b2084f9cdc7261b8355347e588 not found: ID does not exist" Jul 22 02:30:00.477199 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.477165 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-fd9dff84f-67px6"] Jul 22 02:30:00.484001 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:00.483968 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-fd9dff84f-67px6"] Jul 22 02:30:01.627619 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:01.627585 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="266db83d-8440-47b6-a7c7-8fb294a00554" path="/var/lib/kubelet/pods/266db83d-8440-47b6-a7c7-8fb294a00554/volumes" Jul 22 02:30:18.480689 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:18.480645 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d7b6cf945-r86gn" podUID="b6f86fa5-28b5-4a34-a437-28756a33fdf4" containerName="console" containerID="cri-o://3bd528218ca9bcbc67269ac59d246d2f224848f3c955d3e31e20228dddfcd9f1" gracePeriod=15 Jul 22 02:30:18.782660 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:18.782624 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:30:18.910027 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:18.909986 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-serving-cert\") pod \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " Jul 22 02:30:18.910210 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:18.910038 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-oauth-serving-cert\") pod \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " Jul 22 02:30:18.910210 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:18.910064 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-oauth-config\") pod \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " Jul 22 02:30:18.910210 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:18.910107 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-trusted-ca-bundle\") pod \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " Jul 22 02:30:18.910371 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:18.910306 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-service-ca\") pod \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " Jul 22 02:30:18.910371 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:18.910364 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-config\") pod \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " Jul 22 02:30:18.910474 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:18.910440 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5mzm\" (UniqueName: \"kubernetes.io/projected/b6f86fa5-28b5-4a34-a437-28756a33fdf4-kube-api-access-d5mzm\") pod \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\" (UID: \"b6f86fa5-28b5-4a34-a437-28756a33fdf4\") " Jul 22 02:30:18.910474 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:18.910456 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b6f86fa5-28b5-4a34-a437-28756a33fdf4" (UID: "b6f86fa5-28b5-4a34-a437-28756a33fdf4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:30:18.910659 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:18.910619 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b6f86fa5-28b5-4a34-a437-28756a33fdf4" (UID: "b6f86fa5-28b5-4a34-a437-28756a33fdf4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:30:18.910722 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:18.910651 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-config" (OuterVolumeSpecName: "console-config") pod "b6f86fa5-28b5-4a34-a437-28756a33fdf4" (UID: "b6f86fa5-28b5-4a34-a437-28756a33fdf4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:30:18.910722 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:18.910665 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-service-ca" (OuterVolumeSpecName: "service-ca") pod "b6f86fa5-28b5-4a34-a437-28756a33fdf4" (UID: "b6f86fa5-28b5-4a34-a437-28756a33fdf4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:30:18.910722 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:18.910690 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-oauth-serving-cert\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:30:18.912445 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:18.912410 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b6f86fa5-28b5-4a34-a437-28756a33fdf4" (UID: "b6f86fa5-28b5-4a34-a437-28756a33fdf4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:30:18.912564 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:18.912504 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6f86fa5-28b5-4a34-a437-28756a33fdf4-kube-api-access-d5mzm" (OuterVolumeSpecName: "kube-api-access-d5mzm") pod "b6f86fa5-28b5-4a34-a437-28756a33fdf4" (UID: "b6f86fa5-28b5-4a34-a437-28756a33fdf4"). InnerVolumeSpecName "kube-api-access-d5mzm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:30:18.912564 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:18.912523 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b6f86fa5-28b5-4a34-a437-28756a33fdf4" (UID: "b6f86fa5-28b5-4a34-a437-28756a33fdf4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:30:19.011518 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:19.011424 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5mzm\" (UniqueName: \"kubernetes.io/projected/b6f86fa5-28b5-4a34-a437-28756a33fdf4-kube-api-access-d5mzm\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:30:19.011518 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:19.011462 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-serving-cert\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:30:19.011518 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:19.011474 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-oauth-config\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:30:19.011518 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:19.011482 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-trusted-ca-bundle\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:30:19.011518 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:19.011492 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-service-ca\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:30:19.011518 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:19.011499 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6f86fa5-28b5-4a34-a437-28756a33fdf4-console-config\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:30:19.526370 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:19.526336 2579 generic.go:356] "Generic (PLEG): container finished" podID="b6f86fa5-28b5-4a34-a437-28756a33fdf4" containerID="3bd528218ca9bcbc67269ac59d246d2f224848f3c955d3e31e20228dddfcd9f1" exitCode=0 Jul 22 02:30:19.526771 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:19.526378 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d7b6cf945-r86gn" Jul 22 02:30:19.526771 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:19.526417 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d7b6cf945-r86gn" event={"ID":"b6f86fa5-28b5-4a34-a437-28756a33fdf4","Type":"ContainerDied","Data":"3bd528218ca9bcbc67269ac59d246d2f224848f3c955d3e31e20228dddfcd9f1"} Jul 22 02:30:19.526771 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:19.526472 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d7b6cf945-r86gn" event={"ID":"b6f86fa5-28b5-4a34-a437-28756a33fdf4","Type":"ContainerDied","Data":"20e3b3bc40dc171abe9c538c521703c9ead865061bc740d7d6b0ff45357c3244"} Jul 22 02:30:19.526771 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:19.526492 2579 scope.go:117] "RemoveContainer" containerID="3bd528218ca9bcbc67269ac59d246d2f224848f3c955d3e31e20228dddfcd9f1" Jul 22 02:30:19.535308 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:19.535287 2579 scope.go:117] "RemoveContainer" containerID="3bd528218ca9bcbc67269ac59d246d2f224848f3c955d3e31e20228dddfcd9f1" Jul 22 02:30:19.535594 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:30:19.535577 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bd528218ca9bcbc67269ac59d246d2f224848f3c955d3e31e20228dddfcd9f1\": container with ID starting with 3bd528218ca9bcbc67269ac59d246d2f224848f3c955d3e31e20228dddfcd9f1 not found: ID does not exist" containerID="3bd528218ca9bcbc67269ac59d246d2f224848f3c955d3e31e20228dddfcd9f1" Jul 22 02:30:19.535636 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:19.535603 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bd528218ca9bcbc67269ac59d246d2f224848f3c955d3e31e20228dddfcd9f1"} err="failed to get container status \"3bd528218ca9bcbc67269ac59d246d2f224848f3c955d3e31e20228dddfcd9f1\": rpc error: code = NotFound desc = could not find container \"3bd528218ca9bcbc67269ac59d246d2f224848f3c955d3e31e20228dddfcd9f1\": container with ID starting with 3bd528218ca9bcbc67269ac59d246d2f224848f3c955d3e31e20228dddfcd9f1 not found: ID does not exist" Jul 22 02:30:19.549052 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:19.549024 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d7b6cf945-r86gn"] Jul 22 02:30:19.553663 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:19.553640 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d7b6cf945-r86gn"] Jul 22 02:30:19.628222 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:19.628189 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6f86fa5-28b5-4a34-a437-28756a33fdf4" path="/var/lib/kubelet/pods/b6f86fa5-28b5-4a34-a437-28756a33fdf4/volumes" Jul 22 02:30:31.811568 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:31.811524 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:30:44.588869 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:44.588818 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:30:44.660990 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:44.660953 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:30:54.371692 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:54.371607 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:30:54.426522 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:54.426477 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:30:56.583101 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:56.583069 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:30:56.647073 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:30:56.647041 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:31:05.585770 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:05.585731 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65dffcd474-g2cnz"] Jul 22 02:31:05.693501 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:05.693471 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:31:05.794780 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:05.794734 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:31:05.856830 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:05.856737 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:31:08.413278 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:08.413227 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:31:08.513074 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:08.513038 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:31:16.451564 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:16.451522 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:31:28.927320 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:28.927287 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:31:30.610056 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.610003 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-65dffcd474-g2cnz" podUID="3ffd79fc-2d6f-4892-a913-8ba8116c8fc9" containerName="console" containerID="cri-o://87d77a447e6f37ad7e2aa65d6723c628013f1dd41a294c21cfa634495b3f2659" gracePeriod=15 Jul 22 02:31:30.828958 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.828917 2579 generic.go:356] "Generic (PLEG): container finished" podID="3ffd79fc-2d6f-4892-a913-8ba8116c8fc9" containerID="87d77a447e6f37ad7e2aa65d6723c628013f1dd41a294c21cfa634495b3f2659" exitCode=0 Jul 22 02:31:30.829116 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.828974 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65dffcd474-g2cnz" event={"ID":"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9","Type":"ContainerDied","Data":"87d77a447e6f37ad7e2aa65d6723c628013f1dd41a294c21cfa634495b3f2659"} Jul 22 02:31:30.860496 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.860437 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:31:30.898134 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.898103 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-service-ca\") pod \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " Jul 22 02:31:30.898320 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.898155 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-oauth-config\") pod \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " Jul 22 02:31:30.898320 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.898184 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-trusted-ca-bundle\") pod \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " Jul 22 02:31:30.898320 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.898205 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-serving-cert\") pod \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " Jul 22 02:31:30.898320 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.898227 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tdfg\" (UniqueName: \"kubernetes.io/projected/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-kube-api-access-2tdfg\") pod \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " Jul 22 02:31:30.898320 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.898259 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-oauth-serving-cert\") pod \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " Jul 22 02:31:30.898320 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.898284 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-config\") pod \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\" (UID: \"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9\") " Jul 22 02:31:30.898639 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.898485 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-service-ca" (OuterVolumeSpecName: "service-ca") pod "3ffd79fc-2d6f-4892-a913-8ba8116c8fc9" (UID: "3ffd79fc-2d6f-4892-a913-8ba8116c8fc9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:31:30.898791 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.898768 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-config" (OuterVolumeSpecName: "console-config") pod "3ffd79fc-2d6f-4892-a913-8ba8116c8fc9" (UID: "3ffd79fc-2d6f-4892-a913-8ba8116c8fc9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:31:30.898868 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.898777 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3ffd79fc-2d6f-4892-a913-8ba8116c8fc9" (UID: "3ffd79fc-2d6f-4892-a913-8ba8116c8fc9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:31:30.898868 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.898816 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3ffd79fc-2d6f-4892-a913-8ba8116c8fc9" (UID: "3ffd79fc-2d6f-4892-a913-8ba8116c8fc9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:31:30.900475 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.900446 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3ffd79fc-2d6f-4892-a913-8ba8116c8fc9" (UID: "3ffd79fc-2d6f-4892-a913-8ba8116c8fc9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:31:30.900475 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.900466 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-kube-api-access-2tdfg" (OuterVolumeSpecName: "kube-api-access-2tdfg") pod "3ffd79fc-2d6f-4892-a913-8ba8116c8fc9" (UID: "3ffd79fc-2d6f-4892-a913-8ba8116c8fc9"). InnerVolumeSpecName "kube-api-access-2tdfg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:31:30.900644 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.900628 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3ffd79fc-2d6f-4892-a913-8ba8116c8fc9" (UID: "3ffd79fc-2d6f-4892-a913-8ba8116c8fc9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:31:30.999616 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.999582 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-service-ca\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:31:30.999616 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.999610 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-oauth-config\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:31:30.999616 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.999621 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-trusted-ca-bundle\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:31:30.999851 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.999631 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-serving-cert\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:31:30.999851 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.999641 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tdfg\" (UniqueName: \"kubernetes.io/projected/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-kube-api-access-2tdfg\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:31:30.999851 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.999650 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-oauth-serving-cert\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:31:30.999851 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:30.999662 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9-console-config\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:31:31.839397 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:31.839358 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65dffcd474-g2cnz" event={"ID":"3ffd79fc-2d6f-4892-a913-8ba8116c8fc9","Type":"ContainerDied","Data":"05537e7b48d42435c682583daf7416b7e3103d6f583ce3d143ce13e0de45e52b"} Jul 22 02:31:31.839397 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:31.839377 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65dffcd474-g2cnz" Jul 22 02:31:31.839907 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:31.839440 2579 scope.go:117] "RemoveContainer" containerID="87d77a447e6f37ad7e2aa65d6723c628013f1dd41a294c21cfa634495b3f2659" Jul 22 02:31:31.857042 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:31.857016 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65dffcd474-g2cnz"] Jul 22 02:31:31.860418 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:31.860393 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65dffcd474-g2cnz"] Jul 22 02:31:33.268838 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:33.268803 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:31:33.366491 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:33.366461 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:31:33.627691 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:31:33.627660 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ffd79fc-2d6f-4892-a913-8ba8116c8fc9" path="/var/lib/kubelet/pods/3ffd79fc-2d6f-4892-a913-8ba8116c8fc9/volumes" Jul 22 02:32:30.219975 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:30.219863 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:32:31.041332 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:31.041295 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:32:34.004070 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:34.004039 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:32:35.558096 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:35.558057 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:32:38.874059 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:38.874023 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:32:39.007118 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:39.007080 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:32:41.798139 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:41.798104 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:32:41.904173 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:41.904141 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:32:43.944528 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:43.944491 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:32:44.367011 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:44.366981 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:32:52.360538 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:52.360488 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:32:54.931262 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:54.931224 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:32:55.022112 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:55.022087 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:32:55.526397 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:55.526369 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 02:32:55.530859 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:55.530832 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 02:32:55.533361 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:55.533336 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 02:32:55.537884 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:55.537862 2579 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 02:32:55.538453 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:32:55.538439 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 02:33:01.429454 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:01.429418 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:01.529051 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:01.529017 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:03.658811 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:03.658782 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:04.038516 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:04.038428 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:07.282040 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:07.282011 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:15.164130 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:15.164089 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:15.255809 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:15.255776 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:15.707874 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:15.707844 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:16.081907 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.081875 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:16.208809 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.208774 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:16.322707 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.322668 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n"] Jul 22 02:33:16.322993 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.322979 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f86fa5-28b5-4a34-a437-28756a33fdf4" containerName="console" Jul 22 02:33:16.323038 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.322994 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f86fa5-28b5-4a34-a437-28756a33fdf4" containerName="console" Jul 22 02:33:16.323038 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.323006 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ffd79fc-2d6f-4892-a913-8ba8116c8fc9" containerName="console" Jul 22 02:33:16.323038 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.323012 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ffd79fc-2d6f-4892-a913-8ba8116c8fc9" containerName="console" Jul 22 02:33:16.323038 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.323026 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab" containerName="console" Jul 22 02:33:16.323038 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.323032 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab" containerName="console" Jul 22 02:33:16.323038 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.323039 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="266db83d-8440-47b6-a7c7-8fb294a00554" containerName="registry" Jul 22 02:33:16.323209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.323044 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="266db83d-8440-47b6-a7c7-8fb294a00554" containerName="registry" Jul 22 02:33:16.323209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.323094 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="266db83d-8440-47b6-a7c7-8fb294a00554" containerName="registry" Jul 22 02:33:16.323209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.323102 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ffd79fc-2d6f-4892-a913-8ba8116c8fc9" containerName="console" Jul 22 02:33:16.323209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.323108 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7bd1a08-d4ef-4f8d-9b6c-baa25b8af3ab" containerName="console" Jul 22 02:33:16.323209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.323112 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6f86fa5-28b5-4a34-a437-28756a33fdf4" containerName="console" Jul 22 02:33:16.324924 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.324909 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:16.329174 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.329154 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 02:33:16.329291 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.329197 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 02:33:16.329291 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.329280 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 02:33:16.329411 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.329397 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 02:33:16.329503 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.329488 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gwnxn\"" Jul 22 02:33:16.365185 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.365114 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n"] Jul 22 02:33:16.438028 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.437991 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km7cr\" (UniqueName: \"kubernetes.io/projected/9eee98d0-955c-4c42-8b41-c5a7af22abbc-kube-api-access-km7cr\") pod \"opendatahub-operator-controller-manager-86c79b8765-msg5n\" (UID: \"9eee98d0-955c-4c42-8b41-c5a7af22abbc\") " pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:16.438199 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.438044 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9eee98d0-955c-4c42-8b41-c5a7af22abbc-webhook-cert\") pod \"opendatahub-operator-controller-manager-86c79b8765-msg5n\" (UID: \"9eee98d0-955c-4c42-8b41-c5a7af22abbc\") " pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:16.438199 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.438105 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9eee98d0-955c-4c42-8b41-c5a7af22abbc-tmp\") pod \"opendatahub-operator-controller-manager-86c79b8765-msg5n\" (UID: \"9eee98d0-955c-4c42-8b41-c5a7af22abbc\") " pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:16.438199 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.438166 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9eee98d0-955c-4c42-8b41-c5a7af22abbc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86c79b8765-msg5n\" (UID: \"9eee98d0-955c-4c42-8b41-c5a7af22abbc\") " pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:16.438300 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.438257 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9eee98d0-955c-4c42-8b41-c5a7af22abbc-manifests\") pod \"opendatahub-operator-controller-manager-86c79b8765-msg5n\" (UID: \"9eee98d0-955c-4c42-8b41-c5a7af22abbc\") " pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:16.508449 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.508412 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:16.539008 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.538972 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9eee98d0-955c-4c42-8b41-c5a7af22abbc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86c79b8765-msg5n\" (UID: \"9eee98d0-955c-4c42-8b41-c5a7af22abbc\") " pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:16.539168 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.539036 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9eee98d0-955c-4c42-8b41-c5a7af22abbc-manifests\") pod \"opendatahub-operator-controller-manager-86c79b8765-msg5n\" (UID: \"9eee98d0-955c-4c42-8b41-c5a7af22abbc\") " pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:16.539168 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.539064 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km7cr\" (UniqueName: \"kubernetes.io/projected/9eee98d0-955c-4c42-8b41-c5a7af22abbc-kube-api-access-km7cr\") pod \"opendatahub-operator-controller-manager-86c79b8765-msg5n\" (UID: \"9eee98d0-955c-4c42-8b41-c5a7af22abbc\") " pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:16.539168 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.539088 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9eee98d0-955c-4c42-8b41-c5a7af22abbc-webhook-cert\") pod \"opendatahub-operator-controller-manager-86c79b8765-msg5n\" (UID: \"9eee98d0-955c-4c42-8b41-c5a7af22abbc\") " pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:16.539168 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.539113 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9eee98d0-955c-4c42-8b41-c5a7af22abbc-tmp\") pod \"opendatahub-operator-controller-manager-86c79b8765-msg5n\" (UID: \"9eee98d0-955c-4c42-8b41-c5a7af22abbc\") " pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:16.539516 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.539487 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9eee98d0-955c-4c42-8b41-c5a7af22abbc-manifests\") pod \"opendatahub-operator-controller-manager-86c79b8765-msg5n\" (UID: \"9eee98d0-955c-4c42-8b41-c5a7af22abbc\") " pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:16.541461 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.541440 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9eee98d0-955c-4c42-8b41-c5a7af22abbc-tmp\") pod \"opendatahub-operator-controller-manager-86c79b8765-msg5n\" (UID: \"9eee98d0-955c-4c42-8b41-c5a7af22abbc\") " pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:16.541604 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.541588 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9eee98d0-955c-4c42-8b41-c5a7af22abbc-webhook-cert\") pod \"opendatahub-operator-controller-manager-86c79b8765-msg5n\" (UID: \"9eee98d0-955c-4c42-8b41-c5a7af22abbc\") " pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:16.541766 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.541747 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9eee98d0-955c-4c42-8b41-c5a7af22abbc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86c79b8765-msg5n\" (UID: \"9eee98d0-955c-4c42-8b41-c5a7af22abbc\") " pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:16.552914 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.552889 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-km7cr\" (UniqueName: \"kubernetes.io/projected/9eee98d0-955c-4c42-8b41-c5a7af22abbc-kube-api-access-km7cr\") pod \"opendatahub-operator-controller-manager-86c79b8765-msg5n\" (UID: \"9eee98d0-955c-4c42-8b41-c5a7af22abbc\") " pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:16.611894 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.611859 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:16.635168 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.635074 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:16.709633 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.709600 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:16.785827 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.785804 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n"] Jul 22 02:33:16.788023 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:33:16.787992 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eee98d0_955c_4c42_8b41_c5a7af22abbc.slice/crio-4bb60920fd261a661f611fbdfc1233bfab7828fb45f3c09e4dd3fd3c5ce43e5e WatchSource:0}: Error finding container 4bb60920fd261a661f611fbdfc1233bfab7828fb45f3c09e4dd3fd3c5ce43e5e: Status 404 returned error can't find the container with id 4bb60920fd261a661f611fbdfc1233bfab7828fb45f3c09e4dd3fd3c5ce43e5e Jul 22 02:33:16.789847 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.789825 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:33:16.789847 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.789820 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:16.853610 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.853581 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:16.955274 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:16.955188 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:17.061968 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:17.061901 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:17.254887 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:17.254778 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" event={"ID":"9eee98d0-955c-4c42-8b41-c5a7af22abbc","Type":"ContainerStarted","Data":"4bb60920fd261a661f611fbdfc1233bfab7828fb45f3c09e4dd3fd3c5ce43e5e"} Jul 22 02:33:20.270786 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:20.270749 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" event={"ID":"9eee98d0-955c-4c42-8b41-c5a7af22abbc","Type":"ContainerStarted","Data":"7fae560df0e7baec3a04aeb90a4a0b6b4acc5053c7493394f617c70588099425"} Jul 22 02:33:21.276078 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:21.276043 2579 generic.go:356] "Generic (PLEG): container finished" podID="9eee98d0-955c-4c42-8b41-c5a7af22abbc" containerID="7fae560df0e7baec3a04aeb90a4a0b6b4acc5053c7493394f617c70588099425" exitCode=0 Jul 22 02:33:21.276537 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:21.276104 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" event={"ID":"9eee98d0-955c-4c42-8b41-c5a7af22abbc","Type":"ContainerDied","Data":"7fae560df0e7baec3a04aeb90a4a0b6b4acc5053c7493394f617c70588099425"} Jul 22 02:33:22.283624 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:22.283583 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" event={"ID":"9eee98d0-955c-4c42-8b41-c5a7af22abbc","Type":"ContainerStarted","Data":"c714ed6047f0595cd144d2e660b812bbe35c42d98868823bd8606a0e64ecc1bd"} Jul 22 02:33:22.284050 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:22.283726 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:22.309258 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:22.309175 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" podStartSLOduration=2.005817381 podStartE2EDuration="6.309159053s" totalImagesPullingTime="3.231381965s" totalInitContainerRuntime="1.071959707s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:33:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:33:22.307070031 +0000 UTC m=+327.155926862" watchObservedRunningTime="2026-07-22 02:33:22.309159053 +0000 UTC m=+327.158015885" Jul 22 02:33:31.915809 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:31.915769 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:32.071839 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:32.071810 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:33.292295 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:33.292265 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-86c79b8765-msg5n" Jul 22 02:33:35.844193 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:35.844158 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:36.069093 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.069059 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:36.384292 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.384232 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:36.550668 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.550629 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7fd555b8c5-6slpl"] Jul 22 02:33:36.561934 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.561895 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7fd555b8c5-6slpl" Jul 22 02:33:36.566779 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.566749 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 22 02:33:36.567084 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.566861 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-h2gxf\"" Jul 22 02:33:36.567453 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.567432 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 22 02:33:36.591904 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.591869 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7fd555b8c5-6slpl"] Jul 22 02:33:36.710867 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.710780 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83709665-5bae-491c-b361-b0bbf7dba1f4-tmp\") pod \"kube-auth-proxy-7fd555b8c5-6slpl\" (UID: \"83709665-5bae-491c-b361-b0bbf7dba1f4\") " pod="openshift-ingress/kube-auth-proxy-7fd555b8c5-6slpl" Jul 22 02:33:36.711125 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.710898 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtn56\" (UniqueName: \"kubernetes.io/projected/83709665-5bae-491c-b361-b0bbf7dba1f4-kube-api-access-jtn56\") pod \"kube-auth-proxy-7fd555b8c5-6slpl\" (UID: \"83709665-5bae-491c-b361-b0bbf7dba1f4\") " pod="openshift-ingress/kube-auth-proxy-7fd555b8c5-6slpl" Jul 22 02:33:36.711125 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.710994 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83709665-5bae-491c-b361-b0bbf7dba1f4-tls-certs\") pod \"kube-auth-proxy-7fd555b8c5-6slpl\" (UID: \"83709665-5bae-491c-b361-b0bbf7dba1f4\") " pod="openshift-ingress/kube-auth-proxy-7fd555b8c5-6slpl" Jul 22 02:33:36.754579 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.754547 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:36.812300 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.812240 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83709665-5bae-491c-b361-b0bbf7dba1f4-tmp\") pod \"kube-auth-proxy-7fd555b8c5-6slpl\" (UID: \"83709665-5bae-491c-b361-b0bbf7dba1f4\") " pod="openshift-ingress/kube-auth-proxy-7fd555b8c5-6slpl" Jul 22 02:33:36.812474 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.812355 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtn56\" (UniqueName: \"kubernetes.io/projected/83709665-5bae-491c-b361-b0bbf7dba1f4-kube-api-access-jtn56\") pod \"kube-auth-proxy-7fd555b8c5-6slpl\" (UID: \"83709665-5bae-491c-b361-b0bbf7dba1f4\") " pod="openshift-ingress/kube-auth-proxy-7fd555b8c5-6slpl" Jul 22 02:33:36.812474 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.812400 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83709665-5bae-491c-b361-b0bbf7dba1f4-tls-certs\") pod \"kube-auth-proxy-7fd555b8c5-6slpl\" (UID: \"83709665-5bae-491c-b361-b0bbf7dba1f4\") " pod="openshift-ingress/kube-auth-proxy-7fd555b8c5-6slpl" Jul 22 02:33:36.814860 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.814825 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83709665-5bae-491c-b361-b0bbf7dba1f4-tmp\") pod \"kube-auth-proxy-7fd555b8c5-6slpl\" (UID: \"83709665-5bae-491c-b361-b0bbf7dba1f4\") " pod="openshift-ingress/kube-auth-proxy-7fd555b8c5-6slpl" Jul 22 02:33:36.815019 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.814977 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83709665-5bae-491c-b361-b0bbf7dba1f4-tls-certs\") pod \"kube-auth-proxy-7fd555b8c5-6slpl\" (UID: \"83709665-5bae-491c-b361-b0bbf7dba1f4\") " pod="openshift-ingress/kube-auth-proxy-7fd555b8c5-6slpl" Jul 22 02:33:36.829492 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.829461 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtn56\" (UniqueName: \"kubernetes.io/projected/83709665-5bae-491c-b361-b0bbf7dba1f4-kube-api-access-jtn56\") pod \"kube-auth-proxy-7fd555b8c5-6slpl\" (UID: \"83709665-5bae-491c-b361-b0bbf7dba1f4\") " pod="openshift-ingress/kube-auth-proxy-7fd555b8c5-6slpl" Jul 22 02:33:36.872991 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:36.872952 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7fd555b8c5-6slpl" Jul 22 02:33:37.021192 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.021157 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gr7fk"] Jul 22 02:33:37.024333 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.024313 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gr7fk" Jul 22 02:33:37.027294 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.027271 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 22 02:33:37.027423 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.027275 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-xxxtk\"" Jul 22 02:33:37.037622 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.037598 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7fd555b8c5-6slpl"] Jul 22 02:33:37.039678 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:33:37.039648 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83709665_5bae_491c_b361_b0bbf7dba1f4.slice/crio-1a1a20f3e141af9e8c538ff0da5f7f2cdc70bb3bfa7d42de80a1ed7e90f2d526 WatchSource:0}: Error finding container 1a1a20f3e141af9e8c538ff0da5f7f2cdc70bb3bfa7d42de80a1ed7e90f2d526: Status 404 returned error can't find the container with id 1a1a20f3e141af9e8c538ff0da5f7f2cdc70bb3bfa7d42de80a1ed7e90f2d526 Jul 22 02:33:37.043463 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.043439 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gr7fk"] Jul 22 02:33:37.114535 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.114489 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b03853b-c8fb-4fd2-a4d8-26e836673c79-cert\") pod \"odh-model-controller-7998c67dc4-gr7fk\" (UID: \"3b03853b-c8fb-4fd2-a4d8-26e836673c79\") " pod="opendatahub/odh-model-controller-7998c67dc4-gr7fk" Jul 22 02:33:37.114735 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.114542 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brcqv\" (UniqueName: \"kubernetes.io/projected/3b03853b-c8fb-4fd2-a4d8-26e836673c79-kube-api-access-brcqv\") pod \"odh-model-controller-7998c67dc4-gr7fk\" (UID: \"3b03853b-c8fb-4fd2-a4d8-26e836673c79\") " pod="opendatahub/odh-model-controller-7998c67dc4-gr7fk" Jul 22 02:33:37.167856 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.167814 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:37.215098 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.215012 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b03853b-c8fb-4fd2-a4d8-26e836673c79-cert\") pod \"odh-model-controller-7998c67dc4-gr7fk\" (UID: \"3b03853b-c8fb-4fd2-a4d8-26e836673c79\") " pod="opendatahub/odh-model-controller-7998c67dc4-gr7fk" Jul 22 02:33:37.215098 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.215054 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brcqv\" (UniqueName: \"kubernetes.io/projected/3b03853b-c8fb-4fd2-a4d8-26e836673c79-kube-api-access-brcqv\") pod \"odh-model-controller-7998c67dc4-gr7fk\" (UID: \"3b03853b-c8fb-4fd2-a4d8-26e836673c79\") " pod="opendatahub/odh-model-controller-7998c67dc4-gr7fk" Jul 22 02:33:37.215296 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:33:37.215185 2579 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 22 02:33:37.215296 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:33:37.215259 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b03853b-c8fb-4fd2-a4d8-26e836673c79-cert podName:3b03853b-c8fb-4fd2-a4d8-26e836673c79 nodeName:}" failed. No retries permitted until 2026-07-22 02:33:37.715240484 +0000 UTC m=+342.564097297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3b03853b-c8fb-4fd2-a4d8-26e836673c79-cert") pod "odh-model-controller-7998c67dc4-gr7fk" (UID: "3b03853b-c8fb-4fd2-a4d8-26e836673c79") : secret "odh-model-controller-webhook-cert" not found Jul 22 02:33:37.231895 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.231860 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brcqv\" (UniqueName: \"kubernetes.io/projected/3b03853b-c8fb-4fd2-a4d8-26e836673c79-kube-api-access-brcqv\") pod \"odh-model-controller-7998c67dc4-gr7fk\" (UID: \"3b03853b-c8fb-4fd2-a4d8-26e836673c79\") " pod="opendatahub/odh-model-controller-7998c67dc4-gr7fk" Jul 22 02:33:37.257961 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.257909 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:37.359751 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.359716 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7fd555b8c5-6slpl" event={"ID":"83709665-5bae-491c-b361-b0bbf7dba1f4","Type":"ContainerStarted","Data":"1a1a20f3e141af9e8c538ff0da5f7f2cdc70bb3bfa7d42de80a1ed7e90f2d526"} Jul 22 02:33:37.370631 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.370576 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:37.569029 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.568913 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:37.720380 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.720338 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b03853b-c8fb-4fd2-a4d8-26e836673c79-cert\") pod \"odh-model-controller-7998c67dc4-gr7fk\" (UID: \"3b03853b-c8fb-4fd2-a4d8-26e836673c79\") " pod="opendatahub/odh-model-controller-7998c67dc4-gr7fk" Jul 22 02:33:37.722992 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.722937 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b03853b-c8fb-4fd2-a4d8-26e836673c79-cert\") pod \"odh-model-controller-7998c67dc4-gr7fk\" (UID: \"3b03853b-c8fb-4fd2-a4d8-26e836673c79\") " pod="opendatahub/odh-model-controller-7998c67dc4-gr7fk" Jul 22 02:33:37.935158 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:37.935120 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gr7fk" Jul 22 02:33:38.071018 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.067664 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:38.138448 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.138412 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gr7fk"] Jul 22 02:33:38.141803 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:33:38.141771 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b03853b_c8fb_4fd2_a4d8_26e836673c79.slice/crio-ece13e9c3cef142463e4027805f2ee364a5f36beaffb935c898efba76ee086fc WatchSource:0}: Error finding container ece13e9c3cef142463e4027805f2ee364a5f36beaffb935c898efba76ee086fc: Status 404 returned error can't find the container with id ece13e9c3cef142463e4027805f2ee364a5f36beaffb935c898efba76ee086fc Jul 22 02:33:38.158305 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.158266 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qh54n"] Jul 22 02:33:38.161854 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.161813 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-qh54n" Jul 22 02:33:38.165660 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.165628 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-2gmf8\"" Jul 22 02:33:38.166664 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.166640 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 22 02:33:38.175842 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.175763 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qh54n"] Jul 22 02:33:38.325374 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.325276 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1db31f22-8184-45f8-9293-c6d7a7c3a80c-tls-certs\") pod \"model-serving-api-54d89bd5fc-qh54n\" (UID: \"1db31f22-8184-45f8-9293-c6d7a7c3a80c\") " pod="opendatahub/model-serving-api-54d89bd5fc-qh54n" Jul 22 02:33:38.325374 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.325331 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhmz7\" (UniqueName: \"kubernetes.io/projected/1db31f22-8184-45f8-9293-c6d7a7c3a80c-kube-api-access-dhmz7\") pod \"model-serving-api-54d89bd5fc-qh54n\" (UID: \"1db31f22-8184-45f8-9293-c6d7a7c3a80c\") " pod="opendatahub/model-serving-api-54d89bd5fc-qh54n" Jul 22 02:33:38.368539 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.368412 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gr7fk" event={"ID":"3b03853b-c8fb-4fd2-a4d8-26e836673c79","Type":"ContainerStarted","Data":"ece13e9c3cef142463e4027805f2ee364a5f36beaffb935c898efba76ee086fc"} Jul 22 02:33:38.405357 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.405294 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:38.426139 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.426097 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1db31f22-8184-45f8-9293-c6d7a7c3a80c-tls-certs\") pod \"model-serving-api-54d89bd5fc-qh54n\" (UID: \"1db31f22-8184-45f8-9293-c6d7a7c3a80c\") " pod="opendatahub/model-serving-api-54d89bd5fc-qh54n" Jul 22 02:33:38.426325 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.426191 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhmz7\" (UniqueName: \"kubernetes.io/projected/1db31f22-8184-45f8-9293-c6d7a7c3a80c-kube-api-access-dhmz7\") pod \"model-serving-api-54d89bd5fc-qh54n\" (UID: \"1db31f22-8184-45f8-9293-c6d7a7c3a80c\") " pod="opendatahub/model-serving-api-54d89bd5fc-qh54n" Jul 22 02:33:38.426325 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:33:38.426257 2579 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 22 02:33:38.426434 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:33:38.426364 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1db31f22-8184-45f8-9293-c6d7a7c3a80c-tls-certs podName:1db31f22-8184-45f8-9293-c6d7a7c3a80c nodeName:}" failed. No retries permitted until 2026-07-22 02:33:38.926338964 +0000 UTC m=+343.775195780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/1db31f22-8184-45f8-9293-c6d7a7c3a80c-tls-certs") pod "model-serving-api-54d89bd5fc-qh54n" (UID: "1db31f22-8184-45f8-9293-c6d7a7c3a80c") : secret "model-serving-api-tls" not found Jul 22 02:33:38.433623 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.433589 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w"] Jul 22 02:33:38.436220 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.436200 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.439124 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.439100 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 02:33:38.439501 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.439482 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 22 02:33:38.439914 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.439664 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 22 02:33:38.439914 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.439680 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 22 02:33:38.439914 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.439723 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-68hhv\"" Jul 22 02:33:38.453594 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.452938 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w"] Jul 22 02:33:38.454748 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.454670 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhmz7\" (UniqueName: \"kubernetes.io/projected/1db31f22-8184-45f8-9293-c6d7a7c3a80c-kube-api-access-dhmz7\") pod \"model-serving-api-54d89bd5fc-qh54n\" (UID: \"1db31f22-8184-45f8-9293-c6d7a7c3a80c\") " pod="opendatahub/model-serving-api-54d89bd5fc-qh54n" Jul 22 02:33:38.514322 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.514280 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:38.527366 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.527330 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-token\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.527570 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.527428 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/18e4a8df-dd47-4aeb-941c-ace4311f7951-local-certs\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.527570 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.527534 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-cacerts\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.527680 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.527582 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.527680 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.527618 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.527680 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.527645 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7czc\" (UniqueName: \"kubernetes.io/projected/18e4a8df-dd47-4aeb-941c-ace4311f7951-kube-api-access-k7czc\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.527825 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.527724 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.628975 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.628869 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-token\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.629159 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.628996 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/18e4a8df-dd47-4aeb-941c-ace4311f7951-local-certs\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.629159 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.629051 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-cacerts\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.629159 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.629083 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.629159 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.629111 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.629159 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.629136 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7czc\" (UniqueName: \"kubernetes.io/projected/18e4a8df-dd47-4aeb-941c-ace4311f7951-kube-api-access-k7czc\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.629417 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.629184 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.630028 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.629992 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.632777 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.632742 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/18e4a8df-dd47-4aeb-941c-ace4311f7951-local-certs\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.633031 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.633011 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.633126 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.633011 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-cacerts\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.633246 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.633226 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.644672 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.644632 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7czc\" (UniqueName: \"kubernetes.io/projected/18e4a8df-dd47-4aeb-941c-ace4311f7951-kube-api-access-k7czc\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.645135 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.645114 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-token\") pod \"istiod-openshift-gateway-7574459656-8bw6w\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.651140 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.651104 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:38.747419 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.747382 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:38.933795 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.933185 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1db31f22-8184-45f8-9293-c6d7a7c3a80c-tls-certs\") pod \"model-serving-api-54d89bd5fc-qh54n\" (UID: \"1db31f22-8184-45f8-9293-c6d7a7c3a80c\") " pod="opendatahub/model-serving-api-54d89bd5fc-qh54n" Jul 22 02:33:38.939234 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.939197 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1db31f22-8184-45f8-9293-c6d7a7c3a80c-tls-certs\") pod \"model-serving-api-54d89bd5fc-qh54n\" (UID: \"1db31f22-8184-45f8-9293-c6d7a7c3a80c\") " pod="opendatahub/model-serving-api-54d89bd5fc-qh54n" Jul 22 02:33:38.951891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:38.951753 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w"] Jul 22 02:33:39.078829 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:39.078791 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-qh54n" Jul 22 02:33:39.833345 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:33:39.833302 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18e4a8df_dd47_4aeb_941c_ace4311f7951.slice/crio-0e766ab6952f5dac6056462bbe3bce1b80e90fd23c2ae60213d916a4f1e0d743 WatchSource:0}: Error finding container 0e766ab6952f5dac6056462bbe3bce1b80e90fd23c2ae60213d916a4f1e0d743: Status 404 returned error can't find the container with id 0e766ab6952f5dac6056462bbe3bce1b80e90fd23c2ae60213d916a4f1e0d743 Jul 22 02:33:40.043583 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:40.043451 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:40.384841 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:40.384799 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" event={"ID":"18e4a8df-dd47-4aeb-941c-ace4311f7951","Type":"ContainerStarted","Data":"0e766ab6952f5dac6056462bbe3bce1b80e90fd23c2ae60213d916a4f1e0d743"} Jul 22 02:33:41.337697 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:41.337663 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qh54n"] Jul 22 02:33:41.341829 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:33:41.341797 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1db31f22_8184_45f8_9293_c6d7a7c3a80c.slice/crio-bab10a3f0bacec5e96f591b5713a350cfb661231a484f6340c361dcdf02f2971 WatchSource:0}: Error finding container bab10a3f0bacec5e96f591b5713a350cfb661231a484f6340c361dcdf02f2971: Status 404 returned error can't find the container with id bab10a3f0bacec5e96f591b5713a350cfb661231a484f6340c361dcdf02f2971 Jul 22 02:33:41.393125 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:41.393077 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7fd555b8c5-6slpl" event={"ID":"83709665-5bae-491c-b361-b0bbf7dba1f4","Type":"ContainerStarted","Data":"f94df157b4befde2c3bfe21a731ff8a277636ba132af14d15e49e4e1ee412423"} Jul 22 02:33:41.395883 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:41.395851 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gr7fk" event={"ID":"3b03853b-c8fb-4fd2-a4d8-26e836673c79","Type":"ContainerStarted","Data":"2f8cf15a2bf2fae1e5a729abe0560e16bab6b859711cf9e23503d184edc77b05"} Jul 22 02:33:41.396071 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:41.395912 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-gr7fk" Jul 22 02:33:41.397587 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:41.397547 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-qh54n" event={"ID":"1db31f22-8184-45f8-9293-c6d7a7c3a80c","Type":"ContainerStarted","Data":"bab10a3f0bacec5e96f591b5713a350cfb661231a484f6340c361dcdf02f2971"} Jul 22 02:33:41.415285 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:41.415201 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7fd555b8c5-6slpl" podStartSLOduration=1.284386277 podStartE2EDuration="5.414500582s" totalImagesPullingTime="4.130114305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:33:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:33:41.41120418 +0000 UTC m=+346.260061039" watchObservedRunningTime="2026-07-22 02:33:41.414500582 +0000 UTC m=+346.263357416" Jul 22 02:33:41.431634 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:41.430395 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-gr7fk" podStartSLOduration=2.369531896 podStartE2EDuration="5.430374507s" totalImagesPullingTime="3.060842611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:33:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:33:41.428199929 +0000 UTC m=+346.277056753" watchObservedRunningTime="2026-07-22 02:33:41.430374507 +0000 UTC m=+346.279231337" Jul 22 02:33:43.826835 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:43.826786 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:33:43.827266 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:43.826895 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:33:44.417296 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:44.417259 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-qh54n" event={"ID":"1db31f22-8184-45f8-9293-c6d7a7c3a80c","Type":"ContainerStarted","Data":"112d323ef87d1f6ab755ad284ca677bf826786362d65662362bde709a29ba2f1"} Jul 22 02:33:44.417491 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:44.417321 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-qh54n" Jul 22 02:33:44.418809 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:44.418787 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" event={"ID":"18e4a8df-dd47-4aeb-941c-ace4311f7951","Type":"ContainerStarted","Data":"6a94a3ad39cef7a9ccf1f8cc8429a5d092bb85719592d034006431e10bb8f876"} Jul 22 02:33:44.418919 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:44.418889 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:44.420689 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:44.420661 2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-8bw6w container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 02:33:44.420788 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:44.420719 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" podUID="18e4a8df-dd47-4aeb-941c-ace4311f7951" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 02:33:44.452490 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:44.452416 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-qh54n" podStartSLOduration=3.9688819669999997 podStartE2EDuration="6.452396908s" totalImagesPullingTime="2.483514941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:33:44.450131307 +0000 UTC m=+349.298988143" watchObservedRunningTime="2026-07-22 02:33:44.452396908 +0000 UTC m=+349.301253740" Jul 22 02:33:44.488336 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:44.488283 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" podStartSLOduration=2.497759588 podStartE2EDuration="6.488265286s" totalImagesPullingTime="3.990505698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:33:44.486211832 +0000 UTC m=+349.335068665" watchObservedRunningTime="2026-07-22 02:33:44.488265286 +0000 UTC m=+349.337122119" Jul 22 02:33:44.656323 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:44.656284 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:44.759652 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:44.759547 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:45.424242 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:45.424208 2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-8bw6w container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 02:33:45.424995 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:45.424260 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" podUID="18e4a8df-dd47-4aeb-941c-ace4311f7951" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 02:33:45.628559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:45.628518 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:45.822336 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:45.822224 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:45.960196 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:45.960164 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:48.424870 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:48.424786 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:49.185062 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:49.185022 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:49.326791 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:49.326759 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zvl82"] Jul 22 02:33:49.330557 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:49.330538 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zvl82" Jul 22 02:33:49.333896 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:49.333875 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-9dg8b\"" Jul 22 02:33:49.334160 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:49.334145 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 22 02:33:49.357539 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:49.357503 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zvl82"] Jul 22 02:33:49.360028 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:49.359996 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:49.424190 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:49.424151 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr9sb\" (UniqueName: \"kubernetes.io/projected/08508ac3-8c86-4698-901a-9d4da42e2296-kube-api-access-gr9sb\") pod \"kserve-controller-manager-d6f4b65b8-zvl82\" (UID: \"08508ac3-8c86-4698-901a-9d4da42e2296\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zvl82" Jul 22 02:33:49.424374 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:49.424212 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08508ac3-8c86-4698-901a-9d4da42e2296-cert\") pod \"kserve-controller-manager-d6f4b65b8-zvl82\" (UID: \"08508ac3-8c86-4698-901a-9d4da42e2296\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zvl82" Jul 22 02:33:49.485126 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:49.485034 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:49.525219 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:49.525182 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr9sb\" (UniqueName: \"kubernetes.io/projected/08508ac3-8c86-4698-901a-9d4da42e2296-kube-api-access-gr9sb\") pod \"kserve-controller-manager-d6f4b65b8-zvl82\" (UID: \"08508ac3-8c86-4698-901a-9d4da42e2296\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zvl82" Jul 22 02:33:49.525367 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:49.525243 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08508ac3-8c86-4698-901a-9d4da42e2296-cert\") pod \"kserve-controller-manager-d6f4b65b8-zvl82\" (UID: \"08508ac3-8c86-4698-901a-9d4da42e2296\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zvl82" Jul 22 02:33:49.527817 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:49.527798 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08508ac3-8c86-4698-901a-9d4da42e2296-cert\") pod \"kserve-controller-manager-d6f4b65b8-zvl82\" (UID: \"08508ac3-8c86-4698-901a-9d4da42e2296\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zvl82" Jul 22 02:33:49.551671 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:49.551641 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr9sb\" (UniqueName: \"kubernetes.io/projected/08508ac3-8c86-4698-901a-9d4da42e2296-kube-api-access-gr9sb\") pod \"kserve-controller-manager-d6f4b65b8-zvl82\" (UID: \"08508ac3-8c86-4698-901a-9d4da42e2296\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zvl82" Jul 22 02:33:49.626958 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:49.626902 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:49.641711 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:49.641675 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zvl82" Jul 22 02:33:49.819841 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:49.819817 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zvl82"] Jul 22 02:33:49.821157 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:33:49.821132 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08508ac3_8c86_4698_901a_9d4da42e2296.slice/crio-4aaee9bbfa220e9e20d11f3df5e448b278ea684397e728d0fe132502c429d169 WatchSource:0}: Error finding container 4aaee9bbfa220e9e20d11f3df5e448b278ea684397e728d0fe132502c429d169: Status 404 returned error can't find the container with id 4aaee9bbfa220e9e20d11f3df5e448b278ea684397e728d0fe132502c429d169 Jul 22 02:33:50.449672 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:50.449633 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zvl82" event={"ID":"08508ac3-8c86-4698-901a-9d4da42e2296","Type":"ContainerStarted","Data":"4aaee9bbfa220e9e20d11f3df5e448b278ea684397e728d0fe132502c429d169"} Jul 22 02:33:50.604985 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:50.604905 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:50.712721 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:50.712634 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:51.994315 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:51.994272 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w"] Jul 22 02:33:51.994760 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:51.994498 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" podUID="18e4a8df-dd47-4aeb-941c-ace4311f7951" containerName="discovery" containerID="cri-o://6a94a3ad39cef7a9ccf1f8cc8429a5d092bb85719592d034006431e10bb8f876" gracePeriod=30 Jul 22 02:33:52.258120 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.258050 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:52.356555 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.356497 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-csr-ca-configmap\") pod \"18e4a8df-dd47-4aeb-941c-ace4311f7951\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " Jul 22 02:33:52.357073 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.357050 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-kubeconfig\") pod \"18e4a8df-dd47-4aeb-941c-ace4311f7951\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " Jul 22 02:33:52.357171 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.357157 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/18e4a8df-dd47-4aeb-941c-ace4311f7951-local-certs\") pod \"18e4a8df-dd47-4aeb-941c-ace4311f7951\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " Jul 22 02:33:52.357207 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.357182 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-cacerts\") pod \"18e4a8df-dd47-4aeb-941c-ace4311f7951\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " Jul 22 02:33:52.357247 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.357232 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-token\") pod \"18e4a8df-dd47-4aeb-941c-ace4311f7951\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " Jul 22 02:33:52.357311 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.357242 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "18e4a8df-dd47-4aeb-941c-ace4311f7951" (UID: "18e4a8df-dd47-4aeb-941c-ace4311f7951"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:33:52.357311 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.357256 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7czc\" (UniqueName: \"kubernetes.io/projected/18e4a8df-dd47-4aeb-941c-ace4311f7951-kube-api-access-k7czc\") pod \"18e4a8df-dd47-4aeb-941c-ace4311f7951\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " Jul 22 02:33:52.357423 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.357314 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-csr-dns-cert\") pod \"18e4a8df-dd47-4aeb-941c-ace4311f7951\" (UID: \"18e4a8df-dd47-4aeb-941c-ace4311f7951\") " Jul 22 02:33:52.357578 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.357556 2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-csr-ca-configmap\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:33:52.360184 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.360137 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-token" (OuterVolumeSpecName: "istio-token") pod "18e4a8df-dd47-4aeb-941c-ace4311f7951" (UID: "18e4a8df-dd47-4aeb-941c-ace4311f7951"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:33:52.360355 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.360227 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-cacerts" (OuterVolumeSpecName: "cacerts") pod "18e4a8df-dd47-4aeb-941c-ace4311f7951" (UID: "18e4a8df-dd47-4aeb-941c-ace4311f7951"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:33:52.360468 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.360447 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e4a8df-dd47-4aeb-941c-ace4311f7951-local-certs" (OuterVolumeSpecName: "local-certs") pod "18e4a8df-dd47-4aeb-941c-ace4311f7951" (UID: "18e4a8df-dd47-4aeb-941c-ace4311f7951"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:33:52.360554 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.360524 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18e4a8df-dd47-4aeb-941c-ace4311f7951-kube-api-access-k7czc" (OuterVolumeSpecName: "kube-api-access-k7czc") pod "18e4a8df-dd47-4aeb-941c-ace4311f7951" (UID: "18e4a8df-dd47-4aeb-941c-ace4311f7951"). InnerVolumeSpecName "kube-api-access-k7czc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:33:52.360631 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.360609 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "18e4a8df-dd47-4aeb-941c-ace4311f7951" (UID: "18e4a8df-dd47-4aeb-941c-ace4311f7951"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:33:52.360710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.360629 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "18e4a8df-dd47-4aeb-941c-ace4311f7951" (UID: "18e4a8df-dd47-4aeb-941c-ace4311f7951"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:33:52.406834 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.406806 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-gr7fk" Jul 22 02:33:52.458455 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.458420 2579 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-kubeconfig\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:33:52.458455 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.458455 2579 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/18e4a8df-dd47-4aeb-941c-ace4311f7951-local-certs\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:33:52.458455 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.458469 2579 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-cacerts\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:33:52.458761 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.458482 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-token\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:33:52.458761 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.458495 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7czc\" (UniqueName: \"kubernetes.io/projected/18e4a8df-dd47-4aeb-941c-ace4311f7951-kube-api-access-k7czc\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:33:52.458761 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.458511 2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/18e4a8df-dd47-4aeb-941c-ace4311f7951-istio-csr-dns-cert\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:33:52.461074 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.461041 2579 generic.go:356] "Generic (PLEG): container finished" podID="18e4a8df-dd47-4aeb-941c-ace4311f7951" containerID="6a94a3ad39cef7a9ccf1f8cc8429a5d092bb85719592d034006431e10bb8f876" exitCode=0 Jul 22 02:33:52.461227 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.461112 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" Jul 22 02:33:52.461227 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.461133 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" event={"ID":"18e4a8df-dd47-4aeb-941c-ace4311f7951","Type":"ContainerDied","Data":"6a94a3ad39cef7a9ccf1f8cc8429a5d092bb85719592d034006431e10bb8f876"} Jul 22 02:33:52.461227 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.461203 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w" event={"ID":"18e4a8df-dd47-4aeb-941c-ace4311f7951","Type":"ContainerDied","Data":"0e766ab6952f5dac6056462bbe3bce1b80e90fd23c2ae60213d916a4f1e0d743"} Jul 22 02:33:52.461380 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.461232 2579 scope.go:117] "RemoveContainer" containerID="6a94a3ad39cef7a9ccf1f8cc8429a5d092bb85719592d034006431e10bb8f876" Jul 22 02:33:52.463266 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.463232 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zvl82" event={"ID":"08508ac3-8c86-4698-901a-9d4da42e2296","Type":"ContainerStarted","Data":"e1f82973efc33f6b94d51a60e64c9a295f5dbbab8d88ab861c9fb9197ac9bddc"} Jul 22 02:33:52.463391 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.463376 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zvl82" Jul 22 02:33:52.472565 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.472542 2579 scope.go:117] "RemoveContainer" containerID="6a94a3ad39cef7a9ccf1f8cc8429a5d092bb85719592d034006431e10bb8f876" Jul 22 02:33:52.472913 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:33:52.472886 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a94a3ad39cef7a9ccf1f8cc8429a5d092bb85719592d034006431e10bb8f876\": container with ID starting with 6a94a3ad39cef7a9ccf1f8cc8429a5d092bb85719592d034006431e10bb8f876 not found: ID does not exist" containerID="6a94a3ad39cef7a9ccf1f8cc8429a5d092bb85719592d034006431e10bb8f876" Jul 22 02:33:52.473023 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.472930 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a94a3ad39cef7a9ccf1f8cc8429a5d092bb85719592d034006431e10bb8f876"} err="failed to get container status \"6a94a3ad39cef7a9ccf1f8cc8429a5d092bb85719592d034006431e10bb8f876\": rpc error: code = NotFound desc = could not find container \"6a94a3ad39cef7a9ccf1f8cc8429a5d092bb85719592d034006431e10bb8f876\": container with ID starting with 6a94a3ad39cef7a9ccf1f8cc8429a5d092bb85719592d034006431e10bb8f876 not found: ID does not exist" Jul 22 02:33:52.490661 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.490581 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zvl82" podStartSLOduration=1.280602276 podStartE2EDuration="3.490560528s" totalImagesPullingTime="2.209958252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:33:52.490285733 +0000 UTC m=+357.339142635" watchObservedRunningTime="2026-07-22 02:33:52.490560528 +0000 UTC m=+357.339417363" Jul 22 02:33:52.513053 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.512924 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w"] Jul 22 02:33:52.524696 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.524660 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8bw6w"] Jul 22 02:33:52.848703 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.848669 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:52.941852 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:52.941811 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:33:53.628119 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:53.628079 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18e4a8df-dd47-4aeb-941c-ace4311f7951" path="/var/lib/kubelet/pods/18e4a8df-dd47-4aeb-941c-ace4311f7951/volumes" Jul 22 02:33:55.428166 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:33:55.428136 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-qh54n" Jul 22 02:34:23.475012 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:23.474980 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zvl82" Jul 22 02:34:36.554904 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:36.554864 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:34:37.489519 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:37.489468 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:34:37.598146 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:37.598104 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:34:37.603781 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:37.603752 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bhcjx"] Jul 22 02:34:37.604160 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:37.604145 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18e4a8df-dd47-4aeb-941c-ace4311f7951" containerName="discovery" Jul 22 02:34:37.604210 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:37.604162 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="18e4a8df-dd47-4aeb-941c-ace4311f7951" containerName="discovery" Jul 22 02:34:37.604251 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:37.604223 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="18e4a8df-dd47-4aeb-941c-ace4311f7951" containerName="discovery" Jul 22 02:34:37.606271 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:37.606251 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bhcjx" Jul 22 02:34:37.610479 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:37.610449 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 02:34:37.610653 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:37.610481 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 02:34:37.610653 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:37.610515 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-6zn7b\"" Jul 22 02:34:37.628816 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:37.628785 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bhcjx"] Jul 22 02:34:37.645678 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:37.645648 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc9wp\" (UniqueName: \"kubernetes.io/projected/a27cec9b-7e28-4318-bb9d-1f7b720affcb-kube-api-access-hc9wp\") pod \"kuadrant-operator-catalog-bhcjx\" (UID: \"a27cec9b-7e28-4318-bb9d-1f7b720affcb\") " pod="kuadrant-system/kuadrant-operator-catalog-bhcjx" Jul 22 02:34:37.695668 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:37.695628 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:34:37.746979 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:37.746857 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9wp\" (UniqueName: \"kubernetes.io/projected/a27cec9b-7e28-4318-bb9d-1f7b720affcb-kube-api-access-hc9wp\") pod \"kuadrant-operator-catalog-bhcjx\" (UID: \"a27cec9b-7e28-4318-bb9d-1f7b720affcb\") " pod="kuadrant-system/kuadrant-operator-catalog-bhcjx" Jul 22 02:34:37.756015 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:37.755986 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc9wp\" (UniqueName: \"kubernetes.io/projected/a27cec9b-7e28-4318-bb9d-1f7b720affcb-kube-api-access-hc9wp\") pod \"kuadrant-operator-catalog-bhcjx\" (UID: \"a27cec9b-7e28-4318-bb9d-1f7b720affcb\") " pod="kuadrant-system/kuadrant-operator-catalog-bhcjx" Jul 22 02:34:37.917637 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:37.917602 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bhcjx" Jul 22 02:34:37.952618 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:37.952586 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bhcjx"] Jul 22 02:34:38.061389 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:38.061366 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bhcjx"] Jul 22 02:34:38.063713 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:34:38.063688 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda27cec9b_7e28_4318_bb9d_1f7b720affcb.slice/crio-581bd40d0fc5e08dc16b2d4f2fabff61adaf00b8cf307130d66b1e19284c8cfb WatchSource:0}: Error finding container 581bd40d0fc5e08dc16b2d4f2fabff61adaf00b8cf307130d66b1e19284c8cfb: Status 404 returned error can't find the container with id 581bd40d0fc5e08dc16b2d4f2fabff61adaf00b8cf307130d66b1e19284c8cfb Jul 22 02:34:38.169587 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:38.169546 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4bs7r"] Jul 22 02:34:38.172356 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:38.172337 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4bs7r" Jul 22 02:34:38.182157 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:38.182125 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4bs7r"] Jul 22 02:34:38.250627 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:38.250596 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqrkc\" (UniqueName: \"kubernetes.io/projected/47f8f702-2641-4227-a18b-0845dc3e7e47-kube-api-access-bqrkc\") pod \"kuadrant-operator-catalog-4bs7r\" (UID: \"47f8f702-2641-4227-a18b-0845dc3e7e47\") " pod="kuadrant-system/kuadrant-operator-catalog-4bs7r" Jul 22 02:34:38.351626 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:38.351586 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqrkc\" (UniqueName: \"kubernetes.io/projected/47f8f702-2641-4227-a18b-0845dc3e7e47-kube-api-access-bqrkc\") pod \"kuadrant-operator-catalog-4bs7r\" (UID: \"47f8f702-2641-4227-a18b-0845dc3e7e47\") " pod="kuadrant-system/kuadrant-operator-catalog-4bs7r" Jul 22 02:34:38.360931 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:38.360898 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqrkc\" (UniqueName: \"kubernetes.io/projected/47f8f702-2641-4227-a18b-0845dc3e7e47-kube-api-access-bqrkc\") pod \"kuadrant-operator-catalog-4bs7r\" (UID: \"47f8f702-2641-4227-a18b-0845dc3e7e47\") " pod="kuadrant-system/kuadrant-operator-catalog-4bs7r" Jul 22 02:34:38.482885 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:38.482855 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4bs7r" Jul 22 02:34:38.618422 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:38.618380 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4bs7r"] Jul 22 02:34:38.620710 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:34:38.620687 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47f8f702_2641_4227_a18b_0845dc3e7e47.slice/crio-d38c4d0215061025560f4807b3e0d9b65079060f13bd0ce236ee7a1d5962ca64 WatchSource:0}: Error finding container d38c4d0215061025560f4807b3e0d9b65079060f13bd0ce236ee7a1d5962ca64: Status 404 returned error can't find the container with id d38c4d0215061025560f4807b3e0d9b65079060f13bd0ce236ee7a1d5962ca64 Jul 22 02:34:38.683988 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:38.683932 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bhcjx" event={"ID":"a27cec9b-7e28-4318-bb9d-1f7b720affcb","Type":"ContainerStarted","Data":"581bd40d0fc5e08dc16b2d4f2fabff61adaf00b8cf307130d66b1e19284c8cfb"} Jul 22 02:34:38.685191 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:38.685159 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4bs7r" event={"ID":"47f8f702-2641-4227-a18b-0845dc3e7e47","Type":"ContainerStarted","Data":"d38c4d0215061025560f4807b3e0d9b65079060f13bd0ce236ee7a1d5962ca64"} Jul 22 02:34:40.697588 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:40.697546 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bhcjx" event={"ID":"a27cec9b-7e28-4318-bb9d-1f7b720affcb","Type":"ContainerStarted","Data":"2e3ef4dea597149fef8325c5db00a287b46f4114dd13e143faf5b52f9632b786"} Jul 22 02:34:40.698116 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:40.697613 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-bhcjx" podUID="a27cec9b-7e28-4318-bb9d-1f7b720affcb" containerName="registry-server" containerID="cri-o://2e3ef4dea597149fef8325c5db00a287b46f4114dd13e143faf5b52f9632b786" gracePeriod=2 Jul 22 02:34:40.699238 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:40.699214 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4bs7r" event={"ID":"47f8f702-2641-4227-a18b-0845dc3e7e47","Type":"ContainerStarted","Data":"688ec6283a8f5379fd12c98f79e826dc6193fa9ea9fd902aa6ac42bc371a8efd"} Jul 22 02:34:40.716365 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:40.716322 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-bhcjx" podStartSLOduration=1.504788408 podStartE2EDuration="3.71630503s" totalImagesPullingTime="2.211516622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:34:40.714464053 +0000 UTC m=+405.563320887" watchObservedRunningTime="2026-07-22 02:34:40.71630503 +0000 UTC m=+405.565161862" Jul 22 02:34:40.729882 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:40.729834 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4bs7r" podStartSLOduration=1.072024993 podStartE2EDuration="2.729815756s" totalImagesPullingTime="1.657790763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:34:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:34:40.72910795 +0000 UTC m=+405.577964784" watchObservedRunningTime="2026-07-22 02:34:40.729815756 +0000 UTC m=+405.578672588" Jul 22 02:34:40.951170 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:40.951099 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bhcjx" Jul 22 02:34:40.974450 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:40.974413 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hc9wp\" (UniqueName: \"kubernetes.io/projected/a27cec9b-7e28-4318-bb9d-1f7b720affcb-kube-api-access-hc9wp\") pod \"a27cec9b-7e28-4318-bb9d-1f7b720affcb\" (UID: \"a27cec9b-7e28-4318-bb9d-1f7b720affcb\") " Jul 22 02:34:40.976799 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:40.976770 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a27cec9b-7e28-4318-bb9d-1f7b720affcb-kube-api-access-hc9wp" (OuterVolumeSpecName: "kube-api-access-hc9wp") pod "a27cec9b-7e28-4318-bb9d-1f7b720affcb" (UID: "a27cec9b-7e28-4318-bb9d-1f7b720affcb"). InnerVolumeSpecName "kube-api-access-hc9wp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:34:41.075284 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:41.075242 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hc9wp\" (UniqueName: \"kubernetes.io/projected/a27cec9b-7e28-4318-bb9d-1f7b720affcb-kube-api-access-hc9wp\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:34:41.705687 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:41.705646 2579 generic.go:356] "Generic (PLEG): container finished" podID="a27cec9b-7e28-4318-bb9d-1f7b720affcb" containerID="2e3ef4dea597149fef8325c5db00a287b46f4114dd13e143faf5b52f9632b786" exitCode=0 Jul 22 02:34:41.706167 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:41.705704 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bhcjx" Jul 22 02:34:41.706167 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:41.705731 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bhcjx" event={"ID":"a27cec9b-7e28-4318-bb9d-1f7b720affcb","Type":"ContainerDied","Data":"2e3ef4dea597149fef8325c5db00a287b46f4114dd13e143faf5b52f9632b786"} Jul 22 02:34:41.706167 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:41.705781 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bhcjx" event={"ID":"a27cec9b-7e28-4318-bb9d-1f7b720affcb","Type":"ContainerDied","Data":"581bd40d0fc5e08dc16b2d4f2fabff61adaf00b8cf307130d66b1e19284c8cfb"} Jul 22 02:34:41.706167 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:41.705801 2579 scope.go:117] "RemoveContainer" containerID="2e3ef4dea597149fef8325c5db00a287b46f4114dd13e143faf5b52f9632b786" Jul 22 02:34:41.715151 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:41.715130 2579 scope.go:117] "RemoveContainer" containerID="2e3ef4dea597149fef8325c5db00a287b46f4114dd13e143faf5b52f9632b786" Jul 22 02:34:41.715538 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:34:41.715512 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e3ef4dea597149fef8325c5db00a287b46f4114dd13e143faf5b52f9632b786\": container with ID starting with 2e3ef4dea597149fef8325c5db00a287b46f4114dd13e143faf5b52f9632b786 not found: ID does not exist" containerID="2e3ef4dea597149fef8325c5db00a287b46f4114dd13e143faf5b52f9632b786" Jul 22 02:34:41.715583 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:41.715551 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e3ef4dea597149fef8325c5db00a287b46f4114dd13e143faf5b52f9632b786"} err="failed to get container status \"2e3ef4dea597149fef8325c5db00a287b46f4114dd13e143faf5b52f9632b786\": rpc error: code = NotFound desc = could not find container \"2e3ef4dea597149fef8325c5db00a287b46f4114dd13e143faf5b52f9632b786\": container with ID starting with 2e3ef4dea597149fef8325c5db00a287b46f4114dd13e143faf5b52f9632b786 not found: ID does not exist" Jul 22 02:34:41.723012 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:41.722970 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bhcjx"] Jul 22 02:34:41.727846 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:41.727813 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bhcjx"] Jul 22 02:34:43.627289 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:43.627253 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a27cec9b-7e28-4318-bb9d-1f7b720affcb" path="/var/lib/kubelet/pods/a27cec9b-7e28-4318-bb9d-1f7b720affcb/volumes" Jul 22 02:34:48.483135 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:48.483082 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-4bs7r" Jul 22 02:34:48.483135 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:48.483157 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-4bs7r" Jul 22 02:34:48.507316 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:48.507289 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-4bs7r" Jul 22 02:34:48.761871 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:48.761791 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-4bs7r" Jul 22 02:34:50.073771 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.073737 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f4df9d7cf-x5ccv"] Jul 22 02:34:50.074206 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.074149 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a27cec9b-7e28-4318-bb9d-1f7b720affcb" containerName="registry-server" Jul 22 02:34:50.074206 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.074160 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a27cec9b-7e28-4318-bb9d-1f7b720affcb" containerName="registry-server" Jul 22 02:34:50.074281 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.074240 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a27cec9b-7e28-4318-bb9d-1f7b720affcb" containerName="registry-server" Jul 22 02:34:50.077502 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.077475 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.082143 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.082103 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 02:34:50.082933 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.082914 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 02:34:50.083162 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.082964 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 02:34:50.083292 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.082982 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-q7fvk\"" Jul 22 02:34:50.083292 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.083026 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 02:34:50.083456 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.083048 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 02:34:50.086830 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.086810 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 02:34:50.091539 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.091517 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f4df9d7cf-x5ccv"] Jul 22 02:34:50.133900 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.133864 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:34:50.154016 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.153940 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-trusted-ca-bundle\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.154206 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.154025 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zv6f\" (UniqueName: \"kubernetes.io/projected/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-kube-api-access-9zv6f\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.154206 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.154093 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-console-serving-cert\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.154206 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.154118 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-service-ca\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.154206 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.154152 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-oauth-serving-cert\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.154206 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.154196 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-console-config\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.154376 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.154212 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-console-oauth-config\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.255511 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.255467 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-console-serving-cert\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.255511 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.255508 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-service-ca\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.255775 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.255564 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-oauth-serving-cert\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.255775 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.255595 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-console-config\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.255775 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.255618 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-console-oauth-config\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.255775 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.255646 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-trusted-ca-bundle\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.255775 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.255674 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zv6f\" (UniqueName: \"kubernetes.io/projected/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-kube-api-access-9zv6f\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.256428 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.256402 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-console-config\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.256595 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.256572 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-oauth-serving-cert\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.256666 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.256645 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-service-ca\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.256725 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.256651 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-trusted-ca-bundle\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.258692 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.258668 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-console-oauth-config\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.258794 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.258753 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-console-serving-cert\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.265159 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.265134 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zv6f\" (UniqueName: \"kubernetes.io/projected/a0edd0e0-9dbc-4052-a2cc-f87a81fb0782-kube-api-access-9zv6f\") pod \"console-f4df9d7cf-x5ccv\" (UID: \"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782\") " pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.272825 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.272796 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:34:50.373439 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.373401 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:34:50.392058 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.392027 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:34:50.542364 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.542334 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f4df9d7cf-x5ccv"] Jul 22 02:34:50.545131 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:34:50.545101 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0edd0e0_9dbc_4052_a2cc_f87a81fb0782.slice/crio-0d6e1f0311a5207576cb31776dabfccd0670fd6cd0ecedd738878a88e8ce0c66 WatchSource:0}: Error finding container 0d6e1f0311a5207576cb31776dabfccd0670fd6cd0ecedd738878a88e8ce0c66: Status 404 returned error can't find the container with id 0d6e1f0311a5207576cb31776dabfccd0670fd6cd0ecedd738878a88e8ce0c66 Jul 22 02:34:50.750666 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.750574 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f4df9d7cf-x5ccv" event={"ID":"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782","Type":"ContainerStarted","Data":"454e5fb12ffe3b219dc3e4cb07ee16249d3d0406e1b9d4471fa34aa6a0c9a08d"} Jul 22 02:34:50.750666 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.750635 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f4df9d7cf-x5ccv" event={"ID":"a0edd0e0-9dbc-4052-a2cc-f87a81fb0782","Type":"ContainerStarted","Data":"0d6e1f0311a5207576cb31776dabfccd0670fd6cd0ecedd738878a88e8ce0c66"} Jul 22 02:34:50.776112 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.776045 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f4df9d7cf-x5ccv" podStartSLOduration=0.776016901 podStartE2EDuration="776.016901ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:34:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:34:50.774468327 +0000 UTC m=+415.623325159" watchObservedRunningTime="2026-07-22 02:34:50.776016901 +0000 UTC m=+415.624873746" Jul 22 02:34:50.875471 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.875436 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:34:50.945931 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:34:50.945897 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:00.392920 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:00.392886 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:35:00.393349 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:00.392958 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:35:00.397807 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:00.397781 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:35:00.808784 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:00.808703 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f4df9d7cf-x5ccv" Jul 22 02:35:00.997681 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:00.997648 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:01.097754 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:01.097722 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:01.199675 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:01.199642 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:08.661461 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:08.661425 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:09.574180 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:09.574152 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:13.449513 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:13.449469 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cxftt"] Jul 22 02:35:13.454392 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:13.454367 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cxftt" Jul 22 02:35:13.457644 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:13.457622 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-2dm4q\"" Jul 22 02:35:13.472263 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:13.472234 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cxftt"] Jul 22 02:35:13.565716 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:13.565671 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhhkp\" (UniqueName: \"kubernetes.io/projected/4b2e3ecd-4e6d-4e2c-8b69-9c38e2f1e24c-kube-api-access-mhhkp\") pod \"authorino-operator-75cc8d5cb7-cxftt\" (UID: \"4b2e3ecd-4e6d-4e2c-8b69-9c38e2f1e24c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cxftt" Jul 22 02:35:13.666792 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:13.666755 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhhkp\" (UniqueName: \"kubernetes.io/projected/4b2e3ecd-4e6d-4e2c-8b69-9c38e2f1e24c-kube-api-access-mhhkp\") pod \"authorino-operator-75cc8d5cb7-cxftt\" (UID: \"4b2e3ecd-4e6d-4e2c-8b69-9c38e2f1e24c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cxftt" Jul 22 02:35:13.684032 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:13.684005 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhhkp\" (UniqueName: \"kubernetes.io/projected/4b2e3ecd-4e6d-4e2c-8b69-9c38e2f1e24c-kube-api-access-mhhkp\") pod \"authorino-operator-75cc8d5cb7-cxftt\" (UID: \"4b2e3ecd-4e6d-4e2c-8b69-9c38e2f1e24c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cxftt" Jul 22 02:35:13.764904 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:13.764817 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cxftt" Jul 22 02:35:13.917907 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:13.917844 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cxftt"] Jul 22 02:35:13.920519 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:35:13.920476 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b2e3ecd_4e6d_4e2c_8b69_9c38e2f1e24c.slice/crio-e6fa4e879fdfaf9dd03f1770987e977e52bb18cf7cbc7baa9dd17151bce5e758 WatchSource:0}: Error finding container e6fa4e879fdfaf9dd03f1770987e977e52bb18cf7cbc7baa9dd17151bce5e758: Status 404 returned error can't find the container with id e6fa4e879fdfaf9dd03f1770987e977e52bb18cf7cbc7baa9dd17151bce5e758 Jul 22 02:35:14.870140 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:14.870102 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cxftt" event={"ID":"4b2e3ecd-4e6d-4e2c-8b69-9c38e2f1e24c","Type":"ContainerStarted","Data":"e6fa4e879fdfaf9dd03f1770987e977e52bb18cf7cbc7baa9dd17151bce5e758"} Jul 22 02:35:15.176670 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:15.176566 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:16.881876 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:16.881840 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cxftt" event={"ID":"4b2e3ecd-4e6d-4e2c-8b69-9c38e2f1e24c","Type":"ContainerStarted","Data":"3390cb83bafbd401ec95c1167634b30d40c7417ec700f33ac076c48f4a849af4"} Jul 22 02:35:16.882296 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:16.881914 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cxftt" Jul 22 02:35:16.900568 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:16.900509 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cxftt" podStartSLOduration=1.5091264 podStartE2EDuration="3.900491649s" totalImagesPullingTime="2.391365249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:35:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:35:16.89900937 +0000 UTC m=+441.747866206" watchObservedRunningTime="2026-07-22 02:35:16.900491649 +0000 UTC m=+441.749348481" Jul 22 02:35:17.548932 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:17.548895 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz94h"] Jul 22 02:35:17.552221 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:17.552201 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz94h" Jul 22 02:35:17.555492 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:17.555467 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 22 02:35:17.555620 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:17.555520 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-wrbj2\"" Jul 22 02:35:17.573645 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:17.573605 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz94h"] Jul 22 02:35:17.709073 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:17.709033 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zp2j\" (UniqueName: \"kubernetes.io/projected/74fc1862-c7a1-4cef-93ef-3a5e4407687a-kube-api-access-7zp2j\") pod \"dns-operator-controller-manager-864dd78c4c-wz94h\" (UID: \"74fc1862-c7a1-4cef-93ef-3a5e4407687a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz94h" Jul 22 02:35:17.809791 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:17.809697 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zp2j\" (UniqueName: \"kubernetes.io/projected/74fc1862-c7a1-4cef-93ef-3a5e4407687a-kube-api-access-7zp2j\") pod \"dns-operator-controller-manager-864dd78c4c-wz94h\" (UID: \"74fc1862-c7a1-4cef-93ef-3a5e4407687a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz94h" Jul 22 02:35:17.821870 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:17.821840 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zp2j\" (UniqueName: \"kubernetes.io/projected/74fc1862-c7a1-4cef-93ef-3a5e4407687a-kube-api-access-7zp2j\") pod \"dns-operator-controller-manager-864dd78c4c-wz94h\" (UID: \"74fc1862-c7a1-4cef-93ef-3a5e4407687a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz94h" Jul 22 02:35:17.862693 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:17.862659 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz94h" Jul 22 02:35:18.009672 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:18.009628 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz94h"] Jul 22 02:35:18.011993 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:35:18.011960 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74fc1862_c7a1_4cef_93ef_3a5e4407687a.slice/crio-d8932b76626923f5898ce61a9d09f0310c6074914f108f0481a8aeff0e6ba7ff WatchSource:0}: Error finding container d8932b76626923f5898ce61a9d09f0310c6074914f108f0481a8aeff0e6ba7ff: Status 404 returned error can't find the container with id d8932b76626923f5898ce61a9d09f0310c6074914f108f0481a8aeff0e6ba7ff Jul 22 02:35:18.894848 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:18.894813 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz94h" event={"ID":"74fc1862-c7a1-4cef-93ef-3a5e4407687a","Type":"ContainerStarted","Data":"d8932b76626923f5898ce61a9d09f0310c6074914f108f0481a8aeff0e6ba7ff"} Jul 22 02:35:18.970234 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:18.970185 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:20.907310 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:20.907276 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz94h" event={"ID":"74fc1862-c7a1-4cef-93ef-3a5e4407687a","Type":"ContainerStarted","Data":"920d6a0fa9cd7f95bbb01ffe4b6c0494c190c91e5dacbb5ec699d736361dccaa"} Jul 22 02:35:20.907805 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:20.907411 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz94h" Jul 22 02:35:20.930070 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:20.930018 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz94h" podStartSLOduration=1.713441126 podStartE2EDuration="3.929999548s" totalImagesPullingTime="2.216558422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:35:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:35:20.92806876 +0000 UTC m=+445.776925605" watchObservedRunningTime="2026-07-22 02:35:20.929999548 +0000 UTC m=+445.778856460" Jul 22 02:35:21.245981 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:21.245873 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn"] Jul 22 02:35:21.249330 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:21.249307 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" Jul 22 02:35:21.252408 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:21.252389 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-cv7bm\"" Jul 22 02:35:21.267875 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:21.267847 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn"] Jul 22 02:35:21.334380 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:21.334344 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcbgw\" (UniqueName: \"kubernetes.io/projected/03f5e59a-32d1-47a1-9b6f-65c7b7912f99-kube-api-access-wcbgw\") pod \"limitador-operator-controller-manager-859d7b78bd-vjrtn\" (UID: \"03f5e59a-32d1-47a1-9b6f-65c7b7912f99\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" Jul 22 02:35:21.435129 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:21.435088 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcbgw\" (UniqueName: \"kubernetes.io/projected/03f5e59a-32d1-47a1-9b6f-65c7b7912f99-kube-api-access-wcbgw\") pod \"limitador-operator-controller-manager-859d7b78bd-vjrtn\" (UID: \"03f5e59a-32d1-47a1-9b6f-65c7b7912f99\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" Jul 22 02:35:21.444270 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:21.444242 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcbgw\" (UniqueName: \"kubernetes.io/projected/03f5e59a-32d1-47a1-9b6f-65c7b7912f99-kube-api-access-wcbgw\") pod \"limitador-operator-controller-manager-859d7b78bd-vjrtn\" (UID: \"03f5e59a-32d1-47a1-9b6f-65c7b7912f99\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" Jul 22 02:35:21.559488 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:21.559402 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" Jul 22 02:35:21.704240 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:21.704210 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn"] Jul 22 02:35:21.706542 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:35:21.706520 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03f5e59a_32d1_47a1_9b6f_65c7b7912f99.slice/crio-ec5f7f38666468065a991f1b3939fc3f9100417f7041147bad178d129c2e2a1e WatchSource:0}: Error finding container ec5f7f38666468065a991f1b3939fc3f9100417f7041147bad178d129c2e2a1e: Status 404 returned error can't find the container with id ec5f7f38666468065a991f1b3939fc3f9100417f7041147bad178d129c2e2a1e Jul 22 02:35:21.778694 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:21.778656 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:21.913276 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:21.913243 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" event={"ID":"03f5e59a-32d1-47a1-9b6f-65c7b7912f99","Type":"ContainerStarted","Data":"ec5f7f38666468065a991f1b3939fc3f9100417f7041147bad178d129c2e2a1e"} Jul 22 02:35:23.924666 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:23.924627 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" event={"ID":"03f5e59a-32d1-47a1-9b6f-65c7b7912f99","Type":"ContainerStarted","Data":"a68b55f2f175bc39b8751f1da3fe017154ecf0815928c569586e511839ebef70"} Jul 22 02:35:23.925062 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:23.924695 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" Jul 22 02:35:23.944748 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:23.944690 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" podStartSLOduration=1.262738141 podStartE2EDuration="2.944673473s" totalImagesPullingTime="1.681935332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:35:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:35:23.943101345 +0000 UTC m=+448.791958179" watchObservedRunningTime="2026-07-22 02:35:23.944673473 +0000 UTC m=+448.793530305" Jul 22 02:35:26.574691 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:26.574659 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2"] Jul 22 02:35:26.578392 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:26.578370 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:26.580828 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:26.580807 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d4w27\"" Jul 22 02:35:26.591176 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:26.591150 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2"] Jul 22 02:35:26.775519 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:26.775471 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/be1c999e-7175-46d6-a50f-7cc5358c6200-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rfss2\" (UID: \"be1c999e-7175-46d6-a50f-7cc5358c6200\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:26.775866 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:26.775608 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6r6l\" (UniqueName: \"kubernetes.io/projected/be1c999e-7175-46d6-a50f-7cc5358c6200-kube-api-access-h6r6l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rfss2\" (UID: \"be1c999e-7175-46d6-a50f-7cc5358c6200\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:26.876089 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:26.876055 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6r6l\" (UniqueName: \"kubernetes.io/projected/be1c999e-7175-46d6-a50f-7cc5358c6200-kube-api-access-h6r6l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rfss2\" (UID: \"be1c999e-7175-46d6-a50f-7cc5358c6200\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:26.876275 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:26.876179 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/be1c999e-7175-46d6-a50f-7cc5358c6200-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rfss2\" (UID: \"be1c999e-7175-46d6-a50f-7cc5358c6200\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:26.876607 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:26.876586 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/be1c999e-7175-46d6-a50f-7cc5358c6200-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rfss2\" (UID: \"be1c999e-7175-46d6-a50f-7cc5358c6200\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:26.887768 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:26.887742 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6r6l\" (UniqueName: \"kubernetes.io/projected/be1c999e-7175-46d6-a50f-7cc5358c6200-kube-api-access-h6r6l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rfss2\" (UID: \"be1c999e-7175-46d6-a50f-7cc5358c6200\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:26.888114 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:26.888097 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:27.040782 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:27.040749 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2"] Jul 22 02:35:27.043893 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:35:27.043862 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe1c999e_7175_46d6_a50f_7cc5358c6200.slice/crio-ab5453f772e8d606be09ded20af042e47bfac1f6ff9dd98c813adefc7d239325 WatchSource:0}: Error finding container ab5453f772e8d606be09ded20af042e47bfac1f6ff9dd98c813adefc7d239325: Status 404 returned error can't find the container with id ab5453f772e8d606be09ded20af042e47bfac1f6ff9dd98c813adefc7d239325 Jul 22 02:35:27.891146 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:27.891113 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cxftt" Jul 22 02:35:27.948528 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:27.948494 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" event={"ID":"be1c999e-7175-46d6-a50f-7cc5358c6200","Type":"ContainerStarted","Data":"ab5453f772e8d606be09ded20af042e47bfac1f6ff9dd98c813adefc7d239325"} Jul 22 02:35:31.917148 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:31.917104 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wz94h" Jul 22 02:35:32.653422 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:32.653361 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:32.788179 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:32.788128 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:32.982327 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:32.982228 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" event={"ID":"be1c999e-7175-46d6-a50f-7cc5358c6200","Type":"ContainerStarted","Data":"b3d0eac8e56197909142b45e18f2e297516f52c203c01f550786a15918754461"} Jul 22 02:35:32.982702 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:32.982346 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:33.049725 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:33.049669 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" podStartSLOduration=1.437891097 podStartE2EDuration="7.049647186s" totalImagesPullingTime="5.611756089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:35:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:35:33.046404086 +0000 UTC m=+457.895260922" watchObservedRunningTime="2026-07-22 02:35:33.049647186 +0000 UTC m=+457.898504018" Jul 22 02:35:34.043793 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:34.043755 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:34.931723 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:34.931688 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" Jul 22 02:35:35.507089 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:35.507050 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:35.755622 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:35.755588 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:35.910234 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:35.910189 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:37.012229 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:37.012163 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:37.382922 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:37.382884 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:37.483414 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:37.483367 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:41.190062 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:41.190027 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:41.272516 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:41.272472 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:43.990968 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:43.990922 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:45.160798 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.160763 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq"] Jul 22 02:35:45.164530 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.164510 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" Jul 22 02:35:45.185447 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.185409 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq"] Jul 22 02:35:45.249237 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.249202 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhgvz\" (UniqueName: \"kubernetes.io/projected/03f60d86-b2b6-4bb7-adf3-538d6d8e4996-kube-api-access-qhgvz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-f6jjq\" (UID: \"03f60d86-b2b6-4bb7-adf3-538d6d8e4996\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" Jul 22 02:35:45.249237 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.249241 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03f60d86-b2b6-4bb7-adf3-538d6d8e4996-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-f6jjq\" (UID: \"03f60d86-b2b6-4bb7-adf3-538d6d8e4996\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" Jul 22 02:35:45.350569 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.350523 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhgvz\" (UniqueName: \"kubernetes.io/projected/03f60d86-b2b6-4bb7-adf3-538d6d8e4996-kube-api-access-qhgvz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-f6jjq\" (UID: \"03f60d86-b2b6-4bb7-adf3-538d6d8e4996\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" Jul 22 02:35:45.350569 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.350571 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03f60d86-b2b6-4bb7-adf3-538d6d8e4996-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-f6jjq\" (UID: \"03f60d86-b2b6-4bb7-adf3-538d6d8e4996\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" Jul 22 02:35:45.351070 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.351051 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03f60d86-b2b6-4bb7-adf3-538d6d8e4996-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-f6jjq\" (UID: \"03f60d86-b2b6-4bb7-adf3-538d6d8e4996\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" Jul 22 02:35:45.360448 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.360419 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhgvz\" (UniqueName: \"kubernetes.io/projected/03f60d86-b2b6-4bb7-adf3-538d6d8e4996-kube-api-access-qhgvz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-f6jjq\" (UID: \"03f60d86-b2b6-4bb7-adf3-538d6d8e4996\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" Jul 22 02:35:45.475232 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.475141 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" Jul 22 02:35:45.647310 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.647285 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq"] Jul 22 02:35:45.648771 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:35:45.648742 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03f60d86_b2b6_4bb7_adf3_538d6d8e4996.slice/crio-ee7bcdf112ee172911aeaf9abcd1a9520b76300088d925037832e84b2adc26af WatchSource:0}: Error finding container ee7bcdf112ee172911aeaf9abcd1a9520b76300088d925037832e84b2adc26af: Status 404 returned error can't find the container with id ee7bcdf112ee172911aeaf9abcd1a9520b76300088d925037832e84b2adc26af Jul 22 02:35:45.873191 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.873157 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq"] Jul 22 02:35:45.880933 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.880902 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq"] Jul 22 02:35:45.894868 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.894819 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2"] Jul 22 02:35:45.895102 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.895059 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" podUID="be1c999e-7175-46d6-a50f-7cc5358c6200" containerName="manager" containerID="cri-o://b3d0eac8e56197909142b45e18f2e297516f52c203c01f550786a15918754461" gracePeriod=2 Jul 22 02:35:45.902016 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.901978 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m"] Jul 22 02:35:45.903224 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.903195 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03f60d86-b2b6-4bb7-adf3-538d6d8e4996" containerName="manager" Jul 22 02:35:45.903224 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.903225 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="03f60d86-b2b6-4bb7-adf3-538d6d8e4996" containerName="manager" Jul 22 02:35:45.903440 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.903426 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="03f60d86-b2b6-4bb7-adf3-538d6d8e4996" containerName="manager" Jul 22 02:35:45.909097 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.908936 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" Jul 22 02:35:45.916077 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.916029 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2"] Jul 22 02:35:45.922237 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:45.922193 2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"be1c999e-7175-46d6-a50f-7cc5358c6200\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-rfss2\": pods \"kuadrant-operator-controller-manager-8684c9ff49-rfss2\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:45.953502 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.952817 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m"] Jul 22 02:35:45.957063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.957022 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn"] Jul 22 02:35:45.957261 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.957050 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c40b4842-50b9-414c-a478-7fa18da44bb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vrn9m\" (UID: \"c40b4842-50b9-414c-a478-7fa18da44bb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" Jul 22 02:35:45.957423 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.957292 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" podUID="03f5e59a-32d1-47a1-9b6f-65c7b7912f99" containerName="manager" containerID="cri-o://a68b55f2f175bc39b8751f1da3fe017154ecf0815928c569586e511839ebef70" gracePeriod=2 Jul 22 02:35:45.957651 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.957511 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdgmj\" (UniqueName: \"kubernetes.io/projected/c40b4842-50b9-414c-a478-7fa18da44bb8-kube-api-access-xdgmj\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vrn9m\" (UID: \"c40b4842-50b9-414c-a478-7fa18da44bb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" Jul 22 02:35:45.960142 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.960113 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn"] Jul 22 02:35:45.963250 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.963219 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnn6s"] Jul 22 02:35:45.963801 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.963774 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03f5e59a-32d1-47a1-9b6f-65c7b7912f99" containerName="manager" Jul 22 02:35:45.963801 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.963798 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="03f5e59a-32d1-47a1-9b6f-65c7b7912f99" containerName="manager" Jul 22 02:35:45.964006 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.963815 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be1c999e-7175-46d6-a50f-7cc5358c6200" containerName="manager" Jul 22 02:35:45.964006 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.963824 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="be1c999e-7175-46d6-a50f-7cc5358c6200" containerName="manager" Jul 22 02:35:45.964006 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.963962 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="03f5e59a-32d1-47a1-9b6f-65c7b7912f99" containerName="manager" Jul 22 02:35:45.964006 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.963973 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="be1c999e-7175-46d6-a50f-7cc5358c6200" containerName="manager" Jul 22 02:35:45.967511 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.967212 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnn6s" Jul 22 02:35:45.970473 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:45.969861 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfss2\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="be1c999e-7175-46d6-a50f-7cc5358c6200" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:45.972882 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:45.972840 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfss2\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="be1c999e-7175-46d6-a50f-7cc5358c6200" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:45.974267 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:45.974243 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnn6s"] Jul 22 02:35:45.996804 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:45.996756 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vjrtn\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="03f5e59a-32d1-47a1-9b6f-65c7b7912f99" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" Jul 22 02:35:46.054613 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.054580 2579 generic.go:356] "Generic (PLEG): container finished" podID="be1c999e-7175-46d6-a50f-7cc5358c6200" containerID="b3d0eac8e56197909142b45e18f2e297516f52c203c01f550786a15918754461" exitCode=0 Jul 22 02:35:46.056556 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:46.056534 2579 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" container="manager" Jul 22 02:35:46.058845 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:46.058814 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vjrtn\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="03f5e59a-32d1-47a1-9b6f-65c7b7912f99" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" Jul 22 02:35:46.059097 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.059080 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdgmj\" (UniqueName: \"kubernetes.io/projected/c40b4842-50b9-414c-a478-7fa18da44bb8-kube-api-access-xdgmj\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vrn9m\" (UID: \"c40b4842-50b9-414c-a478-7fa18da44bb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" Jul 22 02:35:46.059176 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.059113 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb85c\" (UniqueName: \"kubernetes.io/projected/8a6156f2-efc7-44cb-9f2b-8a42aa13e712-kube-api-access-sb85c\") pod \"limitador-operator-controller-manager-859d7b78bd-wnn6s\" (UID: \"8a6156f2-efc7-44cb-9f2b-8a42aa13e712\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnn6s" Jul 22 02:35:46.059176 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.059154 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c40b4842-50b9-414c-a478-7fa18da44bb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vrn9m\" (UID: \"c40b4842-50b9-414c-a478-7fa18da44bb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" Jul 22 02:35:46.059544 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.059528 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c40b4842-50b9-414c-a478-7fa18da44bb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vrn9m\" (UID: \"c40b4842-50b9-414c-a478-7fa18da44bb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" Jul 22 02:35:46.060886 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:46.060863 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfss2\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="be1c999e-7175-46d6-a50f-7cc5358c6200" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:46.062699 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:46.062679 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-f6jjq\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="03f60d86-b2b6-4bb7-adf3-538d6d8e4996" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" Jul 22 02:35:46.071932 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.071834 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdgmj\" (UniqueName: \"kubernetes.io/projected/c40b4842-50b9-414c-a478-7fa18da44bb8-kube-api-access-xdgmj\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vrn9m\" (UID: \"c40b4842-50b9-414c-a478-7fa18da44bb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" Jul 22 02:35:46.160441 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.160406 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb85c\" (UniqueName: \"kubernetes.io/projected/8a6156f2-efc7-44cb-9f2b-8a42aa13e712-kube-api-access-sb85c\") pod \"limitador-operator-controller-manager-859d7b78bd-wnn6s\" (UID: \"8a6156f2-efc7-44cb-9f2b-8a42aa13e712\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnn6s" Jul 22 02:35:46.173690 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.173655 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb85c\" (UniqueName: \"kubernetes.io/projected/8a6156f2-efc7-44cb-9f2b-8a42aa13e712-kube-api-access-sb85c\") pod \"limitador-operator-controller-manager-859d7b78bd-wnn6s\" (UID: \"8a6156f2-efc7-44cb-9f2b-8a42aa13e712\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnn6s" Jul 22 02:35:46.195343 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.195306 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:46.197763 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:46.197733 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfss2\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="be1c999e-7175-46d6-a50f-7cc5358c6200" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:46.199833 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:46.199801 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-f6jjq\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="03f60d86-b2b6-4bb7-adf3-538d6d8e4996" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" Jul 22 02:35:46.201729 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:46.201694 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vjrtn\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="03f5e59a-32d1-47a1-9b6f-65c7b7912f99" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" Jul 22 02:35:46.204274 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.204253 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" Jul 22 02:35:46.206389 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:46.206367 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfss2\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="be1c999e-7175-46d6-a50f-7cc5358c6200" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:46.208360 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:46.208338 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-f6jjq\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="03f60d86-b2b6-4bb7-adf3-538d6d8e4996" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" Jul 22 02:35:46.210208 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:46.210185 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vjrtn\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="03f5e59a-32d1-47a1-9b6f-65c7b7912f99" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" Jul 22 02:35:46.261710 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.261678 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6r6l\" (UniqueName: \"kubernetes.io/projected/be1c999e-7175-46d6-a50f-7cc5358c6200-kube-api-access-h6r6l\") pod \"be1c999e-7175-46d6-a50f-7cc5358c6200\" (UID: \"be1c999e-7175-46d6-a50f-7cc5358c6200\") " Jul 22 02:35:46.261870 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.261727 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcbgw\" (UniqueName: \"kubernetes.io/projected/03f5e59a-32d1-47a1-9b6f-65c7b7912f99-kube-api-access-wcbgw\") pod \"03f5e59a-32d1-47a1-9b6f-65c7b7912f99\" (UID: \"03f5e59a-32d1-47a1-9b6f-65c7b7912f99\") " Jul 22 02:35:46.261870 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.261777 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/be1c999e-7175-46d6-a50f-7cc5358c6200-extensions-socket-volume\") pod \"be1c999e-7175-46d6-a50f-7cc5358c6200\" (UID: \"be1c999e-7175-46d6-a50f-7cc5358c6200\") " Jul 22 02:35:46.262373 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.262313 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be1c999e-7175-46d6-a50f-7cc5358c6200-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "be1c999e-7175-46d6-a50f-7cc5358c6200" (UID: "be1c999e-7175-46d6-a50f-7cc5358c6200"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:35:46.264011 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.263987 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03f5e59a-32d1-47a1-9b6f-65c7b7912f99-kube-api-access-wcbgw" (OuterVolumeSpecName: "kube-api-access-wcbgw") pod "03f5e59a-32d1-47a1-9b6f-65c7b7912f99" (UID: "03f5e59a-32d1-47a1-9b6f-65c7b7912f99"). InnerVolumeSpecName "kube-api-access-wcbgw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:35:46.264011 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.263999 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be1c999e-7175-46d6-a50f-7cc5358c6200-kube-api-access-h6r6l" (OuterVolumeSpecName: "kube-api-access-h6r6l") pod "be1c999e-7175-46d6-a50f-7cc5358c6200" (UID: "be1c999e-7175-46d6-a50f-7cc5358c6200"). InnerVolumeSpecName "kube-api-access-h6r6l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:35:46.267129 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.267110 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" Jul 22 02:35:46.320803 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.320770 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnn6s" Jul 22 02:35:46.363107 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.363059 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6r6l\" (UniqueName: \"kubernetes.io/projected/be1c999e-7175-46d6-a50f-7cc5358c6200-kube-api-access-h6r6l\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:35:46.363107 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.363097 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcbgw\" (UniqueName: \"kubernetes.io/projected/03f5e59a-32d1-47a1-9b6f-65c7b7912f99-kube-api-access-wcbgw\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:35:46.363387 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.363118 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/be1c999e-7175-46d6-a50f-7cc5358c6200-extensions-socket-volume\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:35:46.433066 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.433016 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m"] Jul 22 02:35:46.436856 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:35:46.436795 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc40b4842_50b9_414c_a478_7fa18da44bb8.slice/crio-7f36f4a2ba32554d833d461d15c3fb205e6e8a44a6cabb4f0470612064319938 WatchSource:0}: Error finding container 7f36f4a2ba32554d833d461d15c3fb205e6e8a44a6cabb4f0470612064319938: Status 404 returned error can't find the container with id 7f36f4a2ba32554d833d461d15c3fb205e6e8a44a6cabb4f0470612064319938 Jul 22 02:35:46.501277 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:46.501249 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnn6s"] Jul 22 02:35:46.503693 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:35:46.503667 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a6156f2_efc7_44cb_9f2b_8a42aa13e712.slice/crio-117c6a0ac8038ca8841a64654084e3e49b318088ec291042d533b67653cb901a WatchSource:0}: Error finding container 117c6a0ac8038ca8841a64654084e3e49b318088ec291042d533b67653cb901a: Status 404 returned error can't find the container with id 117c6a0ac8038ca8841a64654084e3e49b318088ec291042d533b67653cb901a Jul 22 02:35:47.065268 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.065227 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnn6s" event={"ID":"8a6156f2-efc7-44cb-9f2b-8a42aa13e712","Type":"ContainerStarted","Data":"a63f8ebf2a1c383bb1cb45df8907a684ba93bd0222fd793edd1c1d79e97f17a8"} Jul 22 02:35:47.065452 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.065302 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnn6s" event={"ID":"8a6156f2-efc7-44cb-9f2b-8a42aa13e712","Type":"ContainerStarted","Data":"117c6a0ac8038ca8841a64654084e3e49b318088ec291042d533b67653cb901a"} Jul 22 02:35:47.065452 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.065357 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnn6s" Jul 22 02:35:47.067062 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.067037 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" event={"ID":"c40b4842-50b9-414c-a478-7fa18da44bb8","Type":"ContainerStarted","Data":"e8813db657facd7db2f20d3ed92f98c3b7752dec14b0ab3a9ba1f08391838f94"} Jul 22 02:35:47.067168 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.067086 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" event={"ID":"c40b4842-50b9-414c-a478-7fa18da44bb8","Type":"ContainerStarted","Data":"7f36f4a2ba32554d833d461d15c3fb205e6e8a44a6cabb4f0470612064319938"} Jul 22 02:35:47.067168 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.067136 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" Jul 22 02:35:47.067817 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:47.067790 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfss2\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="be1c999e-7175-46d6-a50f-7cc5358c6200" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:47.068455 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.068435 2579 generic.go:356] "Generic (PLEG): container finished" podID="03f5e59a-32d1-47a1-9b6f-65c7b7912f99" containerID="a68b55f2f175bc39b8751f1da3fe017154ecf0815928c569586e511839ebef70" exitCode=0 Jul 22 02:35:47.068556 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.068474 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" Jul 22 02:35:47.068556 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.068543 2579 scope.go:117] "RemoveContainer" containerID="a68b55f2f175bc39b8751f1da3fe017154ecf0815928c569586e511839ebef70" Jul 22 02:35:47.070131 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.070113 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:47.072338 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.072308 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" podUID="03f60d86-b2b6-4bb7-adf3-538d6d8e4996" containerName="manager" containerID="cri-o://5068d79fe425714e0d27a822c5a4537b0673029e935b504c50da6b6123b2ba95" gracePeriod=2 Jul 22 02:35:47.079428 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.079407 2579 scope.go:117] "RemoveContainer" containerID="a68b55f2f175bc39b8751f1da3fe017154ecf0815928c569586e511839ebef70" Jul 22 02:35:47.079752 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:47.079732 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a68b55f2f175bc39b8751f1da3fe017154ecf0815928c569586e511839ebef70\": container with ID starting with a68b55f2f175bc39b8751f1da3fe017154ecf0815928c569586e511839ebef70 not found: ID does not exist" containerID="a68b55f2f175bc39b8751f1da3fe017154ecf0815928c569586e511839ebef70" Jul 22 02:35:47.079821 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.079766 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a68b55f2f175bc39b8751f1da3fe017154ecf0815928c569586e511839ebef70"} err="failed to get container status \"a68b55f2f175bc39b8751f1da3fe017154ecf0815928c569586e511839ebef70\": rpc error: code = NotFound desc = could not find container \"a68b55f2f175bc39b8751f1da3fe017154ecf0815928c569586e511839ebef70\": container with ID starting with a68b55f2f175bc39b8751f1da3fe017154ecf0815928c569586e511839ebef70 not found: ID does not exist" Jul 22 02:35:47.079821 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.079788 2579 scope.go:117] "RemoveContainer" containerID="b3d0eac8e56197909142b45e18f2e297516f52c203c01f550786a15918754461" Jul 22 02:35:47.093167 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.093123 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnn6s" podStartSLOduration=2.093103766 podStartE2EDuration="2.093103766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:35:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:35:47.092499464 +0000 UTC m=+471.941356320" watchObservedRunningTime="2026-07-22 02:35:47.093103766 +0000 UTC m=+471.941960598" Jul 22 02:35:47.095521 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:47.095495 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-f6jjq\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="03f60d86-b2b6-4bb7-adf3-538d6d8e4996" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" Jul 22 02:35:47.098449 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:47.098427 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vjrtn\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="03f5e59a-32d1-47a1-9b6f-65c7b7912f99" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" Jul 22 02:35:47.100375 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:47.100349 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfss2\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="be1c999e-7175-46d6-a50f-7cc5358c6200" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:47.102285 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:47.102262 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-f6jjq\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="03f60d86-b2b6-4bb7-adf3-538d6d8e4996" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" Jul 22 02:35:47.141006 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.140920 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" podStartSLOduration=2.140899863 podStartE2EDuration="2.140899863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:35:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:35:47.139643235 +0000 UTC m=+471.988500068" watchObservedRunningTime="2026-07-22 02:35:47.140899863 +0000 UTC m=+471.989756695" Jul 22 02:35:47.144506 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:47.144471 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vjrtn\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="03f5e59a-32d1-47a1-9b6f-65c7b7912f99" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" Jul 22 02:35:47.328249 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.328224 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" Jul 22 02:35:47.330502 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:47.330476 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfss2\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="be1c999e-7175-46d6-a50f-7cc5358c6200" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfss2" Jul 22 02:35:47.332870 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:47.332845 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-f6jjq\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="03f60d86-b2b6-4bb7-adf3-538d6d8e4996" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" Jul 22 02:35:47.336134 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:47.336106 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vjrtn\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="03f5e59a-32d1-47a1-9b6f-65c7b7912f99" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vjrtn" Jul 22 02:35:47.471654 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.471619 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03f60d86-b2b6-4bb7-adf3-538d6d8e4996-extensions-socket-volume\") pod \"03f60d86-b2b6-4bb7-adf3-538d6d8e4996\" (UID: \"03f60d86-b2b6-4bb7-adf3-538d6d8e4996\") " Jul 22 02:35:47.471812 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.471742 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhgvz\" (UniqueName: \"kubernetes.io/projected/03f60d86-b2b6-4bb7-adf3-538d6d8e4996-kube-api-access-qhgvz\") pod \"03f60d86-b2b6-4bb7-adf3-538d6d8e4996\" (UID: \"03f60d86-b2b6-4bb7-adf3-538d6d8e4996\") " Jul 22 02:35:47.471975 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.471924 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03f60d86-b2b6-4bb7-adf3-538d6d8e4996-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "03f60d86-b2b6-4bb7-adf3-538d6d8e4996" (UID: "03f60d86-b2b6-4bb7-adf3-538d6d8e4996"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:35:47.474012 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.473987 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03f60d86-b2b6-4bb7-adf3-538d6d8e4996-kube-api-access-qhgvz" (OuterVolumeSpecName: "kube-api-access-qhgvz") pod "03f60d86-b2b6-4bb7-adf3-538d6d8e4996" (UID: "03f60d86-b2b6-4bb7-adf3-538d6d8e4996"). InnerVolumeSpecName "kube-api-access-qhgvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:35:47.572994 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.572877 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhgvz\" (UniqueName: \"kubernetes.io/projected/03f60d86-b2b6-4bb7-adf3-538d6d8e4996-kube-api-access-qhgvz\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:35:47.572994 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.572913 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03f60d86-b2b6-4bb7-adf3-538d6d8e4996-extensions-socket-volume\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:35:47.628195 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.628162 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03f5e59a-32d1-47a1-9b6f-65c7b7912f99" path="/var/lib/kubelet/pods/03f5e59a-32d1-47a1-9b6f-65c7b7912f99/volumes" Jul 22 02:35:47.628552 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.628539 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03f60d86-b2b6-4bb7-adf3-538d6d8e4996" path="/var/lib/kubelet/pods/03f60d86-b2b6-4bb7-adf3-538d6d8e4996/volumes" Jul 22 02:35:47.628840 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:47.628829 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be1c999e-7175-46d6-a50f-7cc5358c6200" path="/var/lib/kubelet/pods/be1c999e-7175-46d6-a50f-7cc5358c6200/volumes" Jul 22 02:35:48.090342 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:48.090308 2579 generic.go:356] "Generic (PLEG): container finished" podID="03f60d86-b2b6-4bb7-adf3-538d6d8e4996" containerID="5068d79fe425714e0d27a822c5a4537b0673029e935b504c50da6b6123b2ba95" exitCode=2 Jul 22 02:35:48.090571 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:48.090364 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" Jul 22 02:35:48.090571 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:48.090412 2579 scope.go:117] "RemoveContainer" containerID="5068d79fe425714e0d27a822c5a4537b0673029e935b504c50da6b6123b2ba95" Jul 22 02:35:48.095465 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:48.095431 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-f6jjq\" is forbidden: User \"system:node:ip-10-0-130-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-249.ec2.internal' and this object" podUID="03f60d86-b2b6-4bb7-adf3-538d6d8e4996" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f6jjq" Jul 22 02:35:48.099856 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:48.099834 2579 scope.go:117] "RemoveContainer" containerID="5068d79fe425714e0d27a822c5a4537b0673029e935b504c50da6b6123b2ba95" Jul 22 02:35:48.100183 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:35:48.100160 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5068d79fe425714e0d27a822c5a4537b0673029e935b504c50da6b6123b2ba95\": container with ID starting with 5068d79fe425714e0d27a822c5a4537b0673029e935b504c50da6b6123b2ba95 not found: ID does not exist" containerID="5068d79fe425714e0d27a822c5a4537b0673029e935b504c50da6b6123b2ba95" Jul 22 02:35:48.100256 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:48.100191 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5068d79fe425714e0d27a822c5a4537b0673029e935b504c50da6b6123b2ba95"} err="failed to get container status \"5068d79fe425714e0d27a822c5a4537b0673029e935b504c50da6b6123b2ba95\": rpc error: code = NotFound desc = could not find container \"5068d79fe425714e0d27a822c5a4537b0673029e935b504c50da6b6123b2ba95\": container with ID starting with 5068d79fe425714e0d27a822c5a4537b0673029e935b504c50da6b6123b2ba95 not found: ID does not exist" Jul 22 02:35:49.583930 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:49.583901 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:49.685460 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:49.685419 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:49.976102 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:49.976008 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:50.035720 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:50.035679 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:52.983240 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:52.983209 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:53.046516 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:53.046481 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:54.295080 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:54.295046 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:54.395544 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:54.395509 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:56.569296 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:56.569257 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:56.660053 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:56.660010 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:35:58.092814 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:58.092784 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" Jul 22 02:35:58.093241 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:58.092864 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnn6s" Jul 22 02:35:58.523323 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:58.523290 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw"] Jul 22 02:35:58.530742 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:58.530709 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" Jul 22 02:35:58.550787 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:58.550753 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw"] Jul 22 02:35:58.675630 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:58.675534 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e77582da-db54-4cab-b409-62086772faad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-n6ksw\" (UID: \"e77582da-db54-4cab-b409-62086772faad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" Jul 22 02:35:58.675630 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:58.675610 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcd4z\" (UniqueName: \"kubernetes.io/projected/e77582da-db54-4cab-b409-62086772faad-kube-api-access-qcd4z\") pod \"kuadrant-operator-controller-manager-86878c769f-n6ksw\" (UID: \"e77582da-db54-4cab-b409-62086772faad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" Jul 22 02:35:58.776923 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:58.776880 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e77582da-db54-4cab-b409-62086772faad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-n6ksw\" (UID: \"e77582da-db54-4cab-b409-62086772faad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" Jul 22 02:35:58.777132 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:58.776997 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcd4z\" (UniqueName: \"kubernetes.io/projected/e77582da-db54-4cab-b409-62086772faad-kube-api-access-qcd4z\") pod \"kuadrant-operator-controller-manager-86878c769f-n6ksw\" (UID: \"e77582da-db54-4cab-b409-62086772faad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" Jul 22 02:35:58.777338 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:58.777312 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e77582da-db54-4cab-b409-62086772faad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-n6ksw\" (UID: \"e77582da-db54-4cab-b409-62086772faad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" Jul 22 02:35:58.786018 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:58.785991 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcd4z\" (UniqueName: \"kubernetes.io/projected/e77582da-db54-4cab-b409-62086772faad-kube-api-access-qcd4z\") pod \"kuadrant-operator-controller-manager-86878c769f-n6ksw\" (UID: \"e77582da-db54-4cab-b409-62086772faad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" Jul 22 02:35:58.849135 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:58.849097 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" Jul 22 02:35:59.010434 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:59.010408 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw"] Jul 22 02:35:59.013063 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:35:59.013028 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode77582da_db54_4cab_b409_62086772faad.slice/crio-2e541732b0a5729462a8ae7ccf3fd694a63da77eaff8624ef6f305d5a22871e1 WatchSource:0}: Error finding container 2e541732b0a5729462a8ae7ccf3fd694a63da77eaff8624ef6f305d5a22871e1: Status 404 returned error can't find the container with id 2e541732b0a5729462a8ae7ccf3fd694a63da77eaff8624ef6f305d5a22871e1 Jul 22 02:35:59.154201 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:59.154160 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" event={"ID":"e77582da-db54-4cab-b409-62086772faad","Type":"ContainerStarted","Data":"f423a3408368f3c3d008826828df346ff879ea29baa61e7026eb005d7f8f3c16"} Jul 22 02:35:59.154645 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:59.154216 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" event={"ID":"e77582da-db54-4cab-b409-62086772faad","Type":"ContainerStarted","Data":"2e541732b0a5729462a8ae7ccf3fd694a63da77eaff8624ef6f305d5a22871e1"} Jul 22 02:35:59.154645 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:59.154328 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" Jul 22 02:35:59.186644 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:35:59.186584 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" podStartSLOduration=1.186568939 podStartE2EDuration="1.186568939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:35:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:35:59.18651222 +0000 UTC m=+484.035369053" watchObservedRunningTime="2026-07-22 02:35:59.186568939 +0000 UTC m=+484.035425772" Jul 22 02:36:00.242791 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:00.242753 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:00.343048 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:00.343009 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:08.052356 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:08.052311 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:08.153298 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:08.153254 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:08.453739 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:08.453695 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:08.519272 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:08.519237 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:10.161125 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:10.161093 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" Jul 22 02:36:10.220168 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:10.220132 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m"] Jul 22 02:36:10.220366 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:10.220339 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" podUID="c40b4842-50b9-414c-a478-7fa18da44bb8" containerName="manager" containerID="cri-o://e8813db657facd7db2f20d3ed92f98c3b7752dec14b0ab3a9ba1f08391838f94" gracePeriod=10 Jul 22 02:36:10.481240 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:10.481211 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" Jul 22 02:36:10.578550 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:10.578512 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c40b4842-50b9-414c-a478-7fa18da44bb8-extensions-socket-volume\") pod \"c40b4842-50b9-414c-a478-7fa18da44bb8\" (UID: \"c40b4842-50b9-414c-a478-7fa18da44bb8\") " Jul 22 02:36:10.578740 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:10.578575 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdgmj\" (UniqueName: \"kubernetes.io/projected/c40b4842-50b9-414c-a478-7fa18da44bb8-kube-api-access-xdgmj\") pod \"c40b4842-50b9-414c-a478-7fa18da44bb8\" (UID: \"c40b4842-50b9-414c-a478-7fa18da44bb8\") " Jul 22 02:36:10.578960 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:10.578921 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c40b4842-50b9-414c-a478-7fa18da44bb8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c40b4842-50b9-414c-a478-7fa18da44bb8" (UID: "c40b4842-50b9-414c-a478-7fa18da44bb8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:36:10.580831 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:10.580801 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c40b4842-50b9-414c-a478-7fa18da44bb8-kube-api-access-xdgmj" (OuterVolumeSpecName: "kube-api-access-xdgmj") pod "c40b4842-50b9-414c-a478-7fa18da44bb8" (UID: "c40b4842-50b9-414c-a478-7fa18da44bb8"). InnerVolumeSpecName "kube-api-access-xdgmj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:36:10.680078 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:10.680044 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdgmj\" (UniqueName: \"kubernetes.io/projected/c40b4842-50b9-414c-a478-7fa18da44bb8-kube-api-access-xdgmj\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:36:10.680078 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:10.680073 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c40b4842-50b9-414c-a478-7fa18da44bb8-extensions-socket-volume\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:36:11.116860 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:11.116827 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:11.219474 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:11.219444 2579 generic.go:356] "Generic (PLEG): container finished" podID="c40b4842-50b9-414c-a478-7fa18da44bb8" containerID="e8813db657facd7db2f20d3ed92f98c3b7752dec14b0ab3a9ba1f08391838f94" exitCode=0 Jul 22 02:36:11.219926 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:11.219493 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" Jul 22 02:36:11.219926 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:11.219501 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" event={"ID":"c40b4842-50b9-414c-a478-7fa18da44bb8","Type":"ContainerDied","Data":"e8813db657facd7db2f20d3ed92f98c3b7752dec14b0ab3a9ba1f08391838f94"} Jul 22 02:36:11.219926 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:11.219546 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m" event={"ID":"c40b4842-50b9-414c-a478-7fa18da44bb8","Type":"ContainerDied","Data":"7f36f4a2ba32554d833d461d15c3fb205e6e8a44a6cabb4f0470612064319938"} Jul 22 02:36:11.219926 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:11.219569 2579 scope.go:117] "RemoveContainer" containerID="e8813db657facd7db2f20d3ed92f98c3b7752dec14b0ab3a9ba1f08391838f94" Jul 22 02:36:11.222429 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:11.222392 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:11.231718 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:11.231515 2579 scope.go:117] "RemoveContainer" containerID="e8813db657facd7db2f20d3ed92f98c3b7752dec14b0ab3a9ba1f08391838f94" Jul 22 02:36:11.231866 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:36:11.231843 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8813db657facd7db2f20d3ed92f98c3b7752dec14b0ab3a9ba1f08391838f94\": container with ID starting with e8813db657facd7db2f20d3ed92f98c3b7752dec14b0ab3a9ba1f08391838f94 not found: ID does not exist" containerID="e8813db657facd7db2f20d3ed92f98c3b7752dec14b0ab3a9ba1f08391838f94" Jul 22 02:36:11.231922 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:11.231876 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8813db657facd7db2f20d3ed92f98c3b7752dec14b0ab3a9ba1f08391838f94"} err="failed to get container status \"e8813db657facd7db2f20d3ed92f98c3b7752dec14b0ab3a9ba1f08391838f94\": rpc error: code = NotFound desc = could not find container \"e8813db657facd7db2f20d3ed92f98c3b7752dec14b0ab3a9ba1f08391838f94\": container with ID starting with e8813db657facd7db2f20d3ed92f98c3b7752dec14b0ab3a9ba1f08391838f94 not found: ID does not exist" Jul 22 02:36:11.245891 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:11.245862 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m"] Jul 22 02:36:11.252131 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:11.252099 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vrn9m"] Jul 22 02:36:11.628221 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:11.628187 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c40b4842-50b9-414c-a478-7fa18da44bb8" path="/var/lib/kubelet/pods/c40b4842-50b9-414c-a478-7fa18da44bb8/volumes" Jul 22 02:36:14.565417 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.565379 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22"] Jul 22 02:36:14.565870 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.565787 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c40b4842-50b9-414c-a478-7fa18da44bb8" containerName="manager" Jul 22 02:36:14.565870 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.565800 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c40b4842-50b9-414c-a478-7fa18da44bb8" containerName="manager" Jul 22 02:36:14.566021 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.565878 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c40b4842-50b9-414c-a478-7fa18da44bb8" containerName="manager" Jul 22 02:36:14.568746 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.568728 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.573611 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.573586 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 02:36:14.573758 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.573670 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2l942\"" Jul 22 02:36:14.586228 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.586180 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22"] Jul 22 02:36:14.605330 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.605279 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:14.619752 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.619720 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53ddb162-08b8-45cd-b2ed-d3557318eb6f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.619973 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.619765 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53ddb162-08b8-45cd-b2ed-d3557318eb6f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.619973 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.619801 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53ddb162-08b8-45cd-b2ed-d3557318eb6f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.619973 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.619828 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53ddb162-08b8-45cd-b2ed-d3557318eb6f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.619973 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.619845 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txgdm\" (UniqueName: \"kubernetes.io/projected/53ddb162-08b8-45cd-b2ed-d3557318eb6f-kube-api-access-txgdm\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.619973 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.619863 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53ddb162-08b8-45cd-b2ed-d3557318eb6f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.620206 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.620031 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53ddb162-08b8-45cd-b2ed-d3557318eb6f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.620206 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.620072 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53ddb162-08b8-45cd-b2ed-d3557318eb6f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.620206 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.620135 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53ddb162-08b8-45cd-b2ed-d3557318eb6f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.720855 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.720807 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53ddb162-08b8-45cd-b2ed-d3557318eb6f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.720855 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.720859 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53ddb162-08b8-45cd-b2ed-d3557318eb6f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.721163 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.720892 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53ddb162-08b8-45cd-b2ed-d3557318eb6f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.721163 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.720914 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53ddb162-08b8-45cd-b2ed-d3557318eb6f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.721163 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.720973 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53ddb162-08b8-45cd-b2ed-d3557318eb6f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.721163 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.721002 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txgdm\" (UniqueName: \"kubernetes.io/projected/53ddb162-08b8-45cd-b2ed-d3557318eb6f-kube-api-access-txgdm\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.721163 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.721032 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53ddb162-08b8-45cd-b2ed-d3557318eb6f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.721163 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.721141 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53ddb162-08b8-45cd-b2ed-d3557318eb6f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.721475 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.721171 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53ddb162-08b8-45cd-b2ed-d3557318eb6f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.721475 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.721290 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53ddb162-08b8-45cd-b2ed-d3557318eb6f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.721730 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.721703 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53ddb162-08b8-45cd-b2ed-d3557318eb6f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.721795 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.721780 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53ddb162-08b8-45cd-b2ed-d3557318eb6f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.721850 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.721800 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53ddb162-08b8-45cd-b2ed-d3557318eb6f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.722070 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.722046 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53ddb162-08b8-45cd-b2ed-d3557318eb6f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.723492 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.723468 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53ddb162-08b8-45cd-b2ed-d3557318eb6f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.724269 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.724249 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53ddb162-08b8-45cd-b2ed-d3557318eb6f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.730655 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.730627 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53ddb162-08b8-45cd-b2ed-d3557318eb6f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.730885 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.730860 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txgdm\" (UniqueName: \"kubernetes.io/projected/53ddb162-08b8-45cd-b2ed-d3557318eb6f-kube-api-access-txgdm\") pod \"maas-default-gateway-openshift-default-56654dcf7-flz22\" (UID: \"53ddb162-08b8-45cd-b2ed-d3557318eb6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.880105 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.880064 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:14.934736 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:14.934679 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:15.033826 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:15.033797 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22"] Jul 22 02:36:15.036407 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:36:15.036375 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53ddb162_08b8_45cd_b2ed_d3557318eb6f.slice/crio-46f2887a9dfaafa4c2163eceff7b708b12b00333fe63ba0e75a9bf2099ef54c9 WatchSource:0}: Error finding container 46f2887a9dfaafa4c2163eceff7b708b12b00333fe63ba0e75a9bf2099ef54c9: Status 404 returned error can't find the container with id 46f2887a9dfaafa4c2163eceff7b708b12b00333fe63ba0e75a9bf2099ef54c9 Jul 22 02:36:15.240990 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:15.240863 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" event={"ID":"53ddb162-08b8-45cd-b2ed-d3557318eb6f","Type":"ContainerStarted","Data":"46f2887a9dfaafa4c2163eceff7b708b12b00333fe63ba0e75a9bf2099ef54c9"} Jul 22 02:36:16.253841 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:16.253791 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:16.355741 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:16.355698 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:18.536817 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:18.536766 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:36:18.537377 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:18.537332 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:36:18.537481 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:18.537461 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:36:19.264623 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:19.264583 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" event={"ID":"53ddb162-08b8-45cd-b2ed-d3557318eb6f","Type":"ContainerStarted","Data":"e93fcf983d241d8a1b4d8967bd7ef128fef4b3f47bc029f1d8be0835c28311f9"} Jul 22 02:36:19.284431 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:19.284363 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" podStartSLOduration=1.786334629 podStartE2EDuration="5.284344789s" totalImagesPullingTime="3.49801016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:36:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:36:19.284007622 +0000 UTC m=+504.132864455" watchObservedRunningTime="2026-07-22 02:36:19.284344789 +0000 UTC m=+504.133201622" Jul 22 02:36:19.664575 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:19.664541 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:19.739191 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:19.739145 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:19.880391 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:19.880345 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:19.885650 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:19.885620 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:20.270240 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:20.270199 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:20.271420 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:20.271382 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:36:27.918102 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:27.918067 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:28.031310 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:28.031265 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:28.088073 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:28.088035 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:28.767064 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:28.767028 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:28.838961 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:28.838917 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:28.924668 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:28.924632 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-lb4rg"] Jul 22 02:36:28.927928 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:28.927906 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lb4rg" Jul 22 02:36:28.930522 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:28.930478 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gfzxx\"" Jul 22 02:36:28.932591 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:28.932563 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lb4rg"] Jul 22 02:36:29.049991 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:29.049864 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlk98\" (UniqueName: \"kubernetes.io/projected/b6ddca36-98d1-4c3c-b78c-88c1de7d7700-kube-api-access-jlk98\") pod \"authorino-75d556bd77-lb4rg\" (UID: \"b6ddca36-98d1-4c3c-b78c-88c1de7d7700\") " pod="kuadrant-system/authorino-75d556bd77-lb4rg" Jul 22 02:36:29.150821 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:29.150786 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlk98\" (UniqueName: \"kubernetes.io/projected/b6ddca36-98d1-4c3c-b78c-88c1de7d7700-kube-api-access-jlk98\") pod \"authorino-75d556bd77-lb4rg\" (UID: \"b6ddca36-98d1-4c3c-b78c-88c1de7d7700\") " pod="kuadrant-system/authorino-75d556bd77-lb4rg" Jul 22 02:36:29.160104 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:29.160074 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlk98\" (UniqueName: \"kubernetes.io/projected/b6ddca36-98d1-4c3c-b78c-88c1de7d7700-kube-api-access-jlk98\") pod \"authorino-75d556bd77-lb4rg\" (UID: \"b6ddca36-98d1-4c3c-b78c-88c1de7d7700\") " pod="kuadrant-system/authorino-75d556bd77-lb4rg" Jul 22 02:36:29.238935 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:29.238899 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lb4rg" Jul 22 02:36:29.396507 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:29.396475 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lb4rg"] Jul 22 02:36:29.399259 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:36:29.399226 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6ddca36_98d1_4c3c_b78c_88c1de7d7700.slice/crio-67e5c870e08191ab663dee6001dacf26f33f872588f9153c8b6f51f0c3503cf0 WatchSource:0}: Error finding container 67e5c870e08191ab663dee6001dacf26f33f872588f9153c8b6f51f0c3503cf0: Status 404 returned error can't find the container with id 67e5c870e08191ab663dee6001dacf26f33f872588f9153c8b6f51f0c3503cf0 Jul 22 02:36:30.343033 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:30.342991 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lb4rg" event={"ID":"b6ddca36-98d1-4c3c-b78c-88c1de7d7700","Type":"ContainerStarted","Data":"67e5c870e08191ab663dee6001dacf26f33f872588f9153c8b6f51f0c3503cf0"} Jul 22 02:36:32.355879 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:32.355841 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lb4rg" event={"ID":"b6ddca36-98d1-4c3c-b78c-88c1de7d7700","Type":"ContainerStarted","Data":"bf9ff6e6e9ab13ca55301d1b8081b5b61ff1f2b75910f19945f6aeb1e043732f"} Jul 22 02:36:32.371643 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:32.371591 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-lb4rg" podStartSLOduration=1.553330962 podStartE2EDuration="4.371575463s" totalImagesPullingTime="2.818244501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:36:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:36:32.371053603 +0000 UTC m=+517.219910440" watchObservedRunningTime="2026-07-22 02:36:32.371575463 +0000 UTC m=+517.220432295" Jul 22 02:36:46.777787 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:46.777653 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:46.871486 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:46.871440 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:47.307204 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:47.307159 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:59.330557 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:59.330519 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:36:59.717586 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:36:59.717488 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:01.053203 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.053163 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lb4rg"] Jul 22 02:37:01.053597 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.053346 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-lb4rg" podUID="b6ddca36-98d1-4c3c-b78c-88c1de7d7700" containerName="authorino" containerID="cri-o://bf9ff6e6e9ab13ca55301d1b8081b5b61ff1f2b75910f19945f6aeb1e043732f" gracePeriod=30 Jul 22 02:37:01.272823 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.272785 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:01.325201 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.325178 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lb4rg" Jul 22 02:37:01.342935 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.342899 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:01.343904 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.343879 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlk98\" (UniqueName: \"kubernetes.io/projected/b6ddca36-98d1-4c3c-b78c-88c1de7d7700-kube-api-access-jlk98\") pod \"b6ddca36-98d1-4c3c-b78c-88c1de7d7700\" (UID: \"b6ddca36-98d1-4c3c-b78c-88c1de7d7700\") " Jul 22 02:37:01.346396 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.346365 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6ddca36-98d1-4c3c-b78c-88c1de7d7700-kube-api-access-jlk98" (OuterVolumeSpecName: "kube-api-access-jlk98") pod "b6ddca36-98d1-4c3c-b78c-88c1de7d7700" (UID: "b6ddca36-98d1-4c3c-b78c-88c1de7d7700"). InnerVolumeSpecName "kube-api-access-jlk98". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:37:01.444927 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.444893 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlk98\" (UniqueName: \"kubernetes.io/projected/b6ddca36-98d1-4c3c-b78c-88c1de7d7700-kube-api-access-jlk98\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:37:01.520583 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.520550 2579 generic.go:356] "Generic (PLEG): container finished" podID="b6ddca36-98d1-4c3c-b78c-88c1de7d7700" containerID="bf9ff6e6e9ab13ca55301d1b8081b5b61ff1f2b75910f19945f6aeb1e043732f" exitCode=0 Jul 22 02:37:01.520756 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.520596 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lb4rg" Jul 22 02:37:01.520756 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.520622 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lb4rg" event={"ID":"b6ddca36-98d1-4c3c-b78c-88c1de7d7700","Type":"ContainerDied","Data":"bf9ff6e6e9ab13ca55301d1b8081b5b61ff1f2b75910f19945f6aeb1e043732f"} Jul 22 02:37:01.520756 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.520653 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lb4rg" event={"ID":"b6ddca36-98d1-4c3c-b78c-88c1de7d7700","Type":"ContainerDied","Data":"67e5c870e08191ab663dee6001dacf26f33f872588f9153c8b6f51f0c3503cf0"} Jul 22 02:37:01.520756 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.520670 2579 scope.go:117] "RemoveContainer" containerID="bf9ff6e6e9ab13ca55301d1b8081b5b61ff1f2b75910f19945f6aeb1e043732f" Jul 22 02:37:01.530803 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.530784 2579 scope.go:117] "RemoveContainer" containerID="bf9ff6e6e9ab13ca55301d1b8081b5b61ff1f2b75910f19945f6aeb1e043732f" Jul 22 02:37:01.531187 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:37:01.531133 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf9ff6e6e9ab13ca55301d1b8081b5b61ff1f2b75910f19945f6aeb1e043732f\": container with ID starting with bf9ff6e6e9ab13ca55301d1b8081b5b61ff1f2b75910f19945f6aeb1e043732f not found: ID does not exist" containerID="bf9ff6e6e9ab13ca55301d1b8081b5b61ff1f2b75910f19945f6aeb1e043732f" Jul 22 02:37:01.531296 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.531196 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf9ff6e6e9ab13ca55301d1b8081b5b61ff1f2b75910f19945f6aeb1e043732f"} err="failed to get container status \"bf9ff6e6e9ab13ca55301d1b8081b5b61ff1f2b75910f19945f6aeb1e043732f\": rpc error: code = NotFound desc = could not find container \"bf9ff6e6e9ab13ca55301d1b8081b5b61ff1f2b75910f19945f6aeb1e043732f\": container with ID starting with bf9ff6e6e9ab13ca55301d1b8081b5b61ff1f2b75910f19945f6aeb1e043732f not found: ID does not exist" Jul 22 02:37:01.544579 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.544533 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lb4rg"] Jul 22 02:37:01.548603 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.548574 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lb4rg"] Jul 22 02:37:01.628844 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:01.628803 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6ddca36-98d1-4c3c-b78c-88c1de7d7700" path="/var/lib/kubelet/pods/b6ddca36-98d1-4c3c-b78c-88c1de7d7700/volumes" Jul 22 02:37:06.040189 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.040151 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:06.063993 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.063929 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f56596985-wjsk6"] Jul 22 02:37:06.064414 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.064397 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6ddca36-98d1-4c3c-b78c-88c1de7d7700" containerName="authorino" Jul 22 02:37:06.064515 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.064417 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6ddca36-98d1-4c3c-b78c-88c1de7d7700" containerName="authorino" Jul 22 02:37:06.064580 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.064536 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6ddca36-98d1-4c3c-b78c-88c1de7d7700" containerName="authorino" Jul 22 02:37:06.121991 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.121925 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f56596985-wjsk6"] Jul 22 02:37:06.122183 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.122043 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f56596985-wjsk6" Jul 22 02:37:06.125107 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.125069 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 22 02:37:06.125107 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.125092 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 22 02:37:06.125337 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.125184 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tkqdg\"" Jul 22 02:37:06.149226 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.149193 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:06.180447 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.180401 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6954c8e6-71ee-4831-b565-f7fb2c90f95d-webhook-cert\") pod \"maas-controller-5f56596985-wjsk6\" (UID: \"6954c8e6-71ee-4831-b565-f7fb2c90f95d\") " pod="opendatahub/maas-controller-5f56596985-wjsk6" Jul 22 02:37:06.180613 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.180480 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp2bb\" (UniqueName: \"kubernetes.io/projected/6954c8e6-71ee-4831-b565-f7fb2c90f95d-kube-api-access-hp2bb\") pod \"maas-controller-5f56596985-wjsk6\" (UID: \"6954c8e6-71ee-4831-b565-f7fb2c90f95d\") " pod="opendatahub/maas-controller-5f56596985-wjsk6" Jul 22 02:37:06.220721 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.220689 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:06.249882 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.249848 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6765967b89-grpzj"] Jul 22 02:37:06.271715 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.271680 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6765967b89-grpzj"] Jul 22 02:37:06.271887 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.271815 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6765967b89-grpzj" Jul 22 02:37:06.281081 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.281051 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6954c8e6-71ee-4831-b565-f7fb2c90f95d-webhook-cert\") pod \"maas-controller-5f56596985-wjsk6\" (UID: \"6954c8e6-71ee-4831-b565-f7fb2c90f95d\") " pod="opendatahub/maas-controller-5f56596985-wjsk6" Jul 22 02:37:06.281252 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.281121 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac-webhook-cert\") pod \"maas-controller-6765967b89-grpzj\" (UID: \"3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac\") " pod="opendatahub/maas-controller-6765967b89-grpzj" Jul 22 02:37:06.281252 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.281147 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q24cg\" (UniqueName: \"kubernetes.io/projected/3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac-kube-api-access-q24cg\") pod \"maas-controller-6765967b89-grpzj\" (UID: \"3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac\") " pod="opendatahub/maas-controller-6765967b89-grpzj" Jul 22 02:37:06.281349 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.281280 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp2bb\" (UniqueName: \"kubernetes.io/projected/6954c8e6-71ee-4831-b565-f7fb2c90f95d-kube-api-access-hp2bb\") pod \"maas-controller-5f56596985-wjsk6\" (UID: \"6954c8e6-71ee-4831-b565-f7fb2c90f95d\") " pod="opendatahub/maas-controller-5f56596985-wjsk6" Jul 22 02:37:06.283879 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.283854 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6954c8e6-71ee-4831-b565-f7fb2c90f95d-webhook-cert\") pod \"maas-controller-5f56596985-wjsk6\" (UID: \"6954c8e6-71ee-4831-b565-f7fb2c90f95d\") " pod="opendatahub/maas-controller-5f56596985-wjsk6" Jul 22 02:37:06.293195 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.293089 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:06.293583 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.293561 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp2bb\" (UniqueName: \"kubernetes.io/projected/6954c8e6-71ee-4831-b565-f7fb2c90f95d-kube-api-access-hp2bb\") pod \"maas-controller-5f56596985-wjsk6\" (UID: \"6954c8e6-71ee-4831-b565-f7fb2c90f95d\") " pod="opendatahub/maas-controller-5f56596985-wjsk6" Jul 22 02:37:06.382447 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.382405 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac-webhook-cert\") pod \"maas-controller-6765967b89-grpzj\" (UID: \"3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac\") " pod="opendatahub/maas-controller-6765967b89-grpzj" Jul 22 02:37:06.382447 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.382449 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q24cg\" (UniqueName: \"kubernetes.io/projected/3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac-kube-api-access-q24cg\") pod \"maas-controller-6765967b89-grpzj\" (UID: \"3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac\") " pod="opendatahub/maas-controller-6765967b89-grpzj" Jul 22 02:37:06.385302 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.385271 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac-webhook-cert\") pod \"maas-controller-6765967b89-grpzj\" (UID: \"3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac\") " pod="opendatahub/maas-controller-6765967b89-grpzj" Jul 22 02:37:06.392020 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.391981 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q24cg\" (UniqueName: \"kubernetes.io/projected/3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac-kube-api-access-q24cg\") pod \"maas-controller-6765967b89-grpzj\" (UID: \"3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac\") " pod="opendatahub/maas-controller-6765967b89-grpzj" Jul 22 02:37:06.433640 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.433599 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f56596985-wjsk6" Jul 22 02:37:06.583080 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.583046 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6765967b89-grpzj" Jul 22 02:37:06.613119 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.613093 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f56596985-wjsk6"] Jul 22 02:37:06.615988 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:37:06.615920 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6954c8e6_71ee_4831_b565_f7fb2c90f95d.slice/crio-14f9e1d8d3d58dc946746083b26cf52978c38df2c9a11c752ae26610151f116f WatchSource:0}: Error finding container 14f9e1d8d3d58dc946746083b26cf52978c38df2c9a11c752ae26610151f116f: Status 404 returned error can't find the container with id 14f9e1d8d3d58dc946746083b26cf52978c38df2c9a11c752ae26610151f116f Jul 22 02:37:06.729262 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:06.729233 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6765967b89-grpzj"] Jul 22 02:37:06.731496 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:37:06.731463 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cf0ed33_91f5_4ac6_a667_b08fe8bb03ac.slice/crio-c0bc13661a968b57770d45ca12e836151096152c59bad4f0b2692083f3c98e70 WatchSource:0}: Error finding container c0bc13661a968b57770d45ca12e836151096152c59bad4f0b2692083f3c98e70: Status 404 returned error can't find the container with id c0bc13661a968b57770d45ca12e836151096152c59bad4f0b2692083f3c98e70 Jul 22 02:37:07.557852 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:07.557749 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6765967b89-grpzj" event={"ID":"3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac","Type":"ContainerStarted","Data":"c0bc13661a968b57770d45ca12e836151096152c59bad4f0b2692083f3c98e70"} Jul 22 02:37:07.560432 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:07.560399 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f56596985-wjsk6" event={"ID":"6954c8e6-71ee-4831-b565-f7fb2c90f95d","Type":"ContainerStarted","Data":"14f9e1d8d3d58dc946746083b26cf52978c38df2c9a11c752ae26610151f116f"} Jul 22 02:37:09.403299 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:09.403255 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:09.511836 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:09.511740 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:09.572907 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:09.572867 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f56596985-wjsk6" event={"ID":"6954c8e6-71ee-4831-b565-f7fb2c90f95d","Type":"ContainerStarted","Data":"b84f5817e5bbbb56097aaf5c36e02f908c02df4c119e60a8fcc51e999b86e267"} Jul 22 02:37:09.573140 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:09.573003 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f56596985-wjsk6" Jul 22 02:37:09.574534 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:09.574504 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6765967b89-grpzj" event={"ID":"3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac","Type":"ContainerStarted","Data":"75a64182f35a7f5f496bc08a41ce800da3e3470d51ead5ae9e10c94d7d11c007"} Jul 22 02:37:09.574677 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:09.574566 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6765967b89-grpzj" Jul 22 02:37:09.604321 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:09.604260 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f56596985-wjsk6" podStartSLOduration=1.027055235 podStartE2EDuration="3.60424211s" totalImagesPullingTime="2.577186875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:37:09.602035393 +0000 UTC m=+554.450892229" watchObservedRunningTime="2026-07-22 02:37:09.60424211 +0000 UTC m=+554.453098946" Jul 22 02:37:09.641929 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:09.641873 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6765967b89-grpzj" podStartSLOduration=1.167888511 podStartE2EDuration="3.641854073s" totalImagesPullingTime="2.473965562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:37:09.635977569 +0000 UTC m=+554.484834399" watchObservedRunningTime="2026-07-22 02:37:09.641854073 +0000 UTC m=+554.490710904" Jul 22 02:37:09.897403 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:09.897368 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:13.235234 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:13.235201 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:14.315813 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:14.315771 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:20.583276 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:20.583236 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6765967b89-grpzj" Jul 22 02:37:20.583888 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:20.583331 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f56596985-wjsk6" Jul 22 02:37:20.661639 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:20.661598 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f56596985-wjsk6"] Jul 22 02:37:20.662252 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:20.661835 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5f56596985-wjsk6" podUID="6954c8e6-71ee-4831-b565-f7fb2c90f95d" containerName="manager" containerID="cri-o://b84f5817e5bbbb56097aaf5c36e02f908c02df4c119e60a8fcc51e999b86e267" gracePeriod=30 Jul 22 02:37:20.922696 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:20.922668 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f56596985-wjsk6" Jul 22 02:37:21.019348 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:21.019312 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hp2bb\" (UniqueName: \"kubernetes.io/projected/6954c8e6-71ee-4831-b565-f7fb2c90f95d-kube-api-access-hp2bb\") pod \"6954c8e6-71ee-4831-b565-f7fb2c90f95d\" (UID: \"6954c8e6-71ee-4831-b565-f7fb2c90f95d\") " Jul 22 02:37:21.019535 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:21.019411 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6954c8e6-71ee-4831-b565-f7fb2c90f95d-webhook-cert\") pod \"6954c8e6-71ee-4831-b565-f7fb2c90f95d\" (UID: \"6954c8e6-71ee-4831-b565-f7fb2c90f95d\") " Jul 22 02:37:21.021862 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:21.021824 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6954c8e6-71ee-4831-b565-f7fb2c90f95d-kube-api-access-hp2bb" (OuterVolumeSpecName: "kube-api-access-hp2bb") pod "6954c8e6-71ee-4831-b565-f7fb2c90f95d" (UID: "6954c8e6-71ee-4831-b565-f7fb2c90f95d"). InnerVolumeSpecName "kube-api-access-hp2bb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:37:21.022027 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:21.021824 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6954c8e6-71ee-4831-b565-f7fb2c90f95d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6954c8e6-71ee-4831-b565-f7fb2c90f95d" (UID: "6954c8e6-71ee-4831-b565-f7fb2c90f95d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:37:21.120503 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:21.120462 2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6954c8e6-71ee-4831-b565-f7fb2c90f95d-webhook-cert\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:37:21.120503 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:21.120497 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hp2bb\" (UniqueName: \"kubernetes.io/projected/6954c8e6-71ee-4831-b565-f7fb2c90f95d-kube-api-access-hp2bb\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:37:21.641886 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:21.641853 2579 generic.go:356] "Generic (PLEG): container finished" podID="6954c8e6-71ee-4831-b565-f7fb2c90f95d" containerID="b84f5817e5bbbb56097aaf5c36e02f908c02df4c119e60a8fcc51e999b86e267" exitCode=0 Jul 22 02:37:21.642287 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:21.641893 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f56596985-wjsk6" Jul 22 02:37:21.642287 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:21.641931 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f56596985-wjsk6" event={"ID":"6954c8e6-71ee-4831-b565-f7fb2c90f95d","Type":"ContainerDied","Data":"b84f5817e5bbbb56097aaf5c36e02f908c02df4c119e60a8fcc51e999b86e267"} Jul 22 02:37:21.642287 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:21.642002 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f56596985-wjsk6" event={"ID":"6954c8e6-71ee-4831-b565-f7fb2c90f95d","Type":"ContainerDied","Data":"14f9e1d8d3d58dc946746083b26cf52978c38df2c9a11c752ae26610151f116f"} Jul 22 02:37:21.642287 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:21.642030 2579 scope.go:117] "RemoveContainer" containerID="b84f5817e5bbbb56097aaf5c36e02f908c02df4c119e60a8fcc51e999b86e267" Jul 22 02:37:21.651679 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:21.651659 2579 scope.go:117] "RemoveContainer" containerID="b84f5817e5bbbb56097aaf5c36e02f908c02df4c119e60a8fcc51e999b86e267" Jul 22 02:37:21.652041 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:37:21.652022 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b84f5817e5bbbb56097aaf5c36e02f908c02df4c119e60a8fcc51e999b86e267\": container with ID starting with b84f5817e5bbbb56097aaf5c36e02f908c02df4c119e60a8fcc51e999b86e267 not found: ID does not exist" containerID="b84f5817e5bbbb56097aaf5c36e02f908c02df4c119e60a8fcc51e999b86e267" Jul 22 02:37:21.652122 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:21.652050 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b84f5817e5bbbb56097aaf5c36e02f908c02df4c119e60a8fcc51e999b86e267"} err="failed to get container status \"b84f5817e5bbbb56097aaf5c36e02f908c02df4c119e60a8fcc51e999b86e267\": rpc error: code = NotFound desc = could not find container \"b84f5817e5bbbb56097aaf5c36e02f908c02df4c119e60a8fcc51e999b86e267\": container with ID starting with b84f5817e5bbbb56097aaf5c36e02f908c02df4c119e60a8fcc51e999b86e267 not found: ID does not exist" Jul 22 02:37:21.660318 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:21.660280 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f56596985-wjsk6"] Jul 22 02:37:21.672465 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:21.668792 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f56596985-wjsk6"] Jul 22 02:37:22.470324 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:22.470284 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:22.528659 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:22.528622 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:23.629129 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:23.629093 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6954c8e6-71ee-4831-b565-f7fb2c90f95d" path="/var/lib/kubelet/pods/6954c8e6-71ee-4831-b565-f7fb2c90f95d/volumes" Jul 22 02:37:42.726011 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:42.725973 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:43.938439 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:43.938402 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-zzvzx"] Jul 22 02:37:43.938831 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:43.938788 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6954c8e6-71ee-4831-b565-f7fb2c90f95d" containerName="manager" Jul 22 02:37:43.938831 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:43.938798 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6954c8e6-71ee-4831-b565-f7fb2c90f95d" containerName="manager" Jul 22 02:37:43.938908 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:43.938876 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6954c8e6-71ee-4831-b565-f7fb2c90f95d" containerName="manager" Jul 22 02:37:43.947241 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:43.947200 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-zzvzx" Jul 22 02:37:43.949909 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:43.949883 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 22 02:37:43.950664 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:43.950634 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-zzvzx"] Jul 22 02:37:44.036305 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:44.036246 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0d891a3-b1ac-468b-b4de-bf3faa5ad570-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-zzvzx\" (UID: \"a0d891a3-b1ac-468b-b4de-bf3faa5ad570\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zzvzx" Jul 22 02:37:44.036305 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:44.036306 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpw57\" (UniqueName: \"kubernetes.io/projected/a0d891a3-b1ac-468b-b4de-bf3faa5ad570-kube-api-access-xpw57\") pod \"payload-pre-processing-7677b774bb-zzvzx\" (UID: \"a0d891a3-b1ac-468b-b4de-bf3faa5ad570\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zzvzx" Jul 22 02:37:44.103826 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:44.103788 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:44.137636 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:44.137595 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0d891a3-b1ac-468b-b4de-bf3faa5ad570-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-zzvzx\" (UID: \"a0d891a3-b1ac-468b-b4de-bf3faa5ad570\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zzvzx" Jul 22 02:37:44.137636 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:44.137634 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpw57\" (UniqueName: \"kubernetes.io/projected/a0d891a3-b1ac-468b-b4de-bf3faa5ad570-kube-api-access-xpw57\") pod \"payload-pre-processing-7677b774bb-zzvzx\" (UID: \"a0d891a3-b1ac-468b-b4de-bf3faa5ad570\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zzvzx" Jul 22 02:37:44.138314 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:44.138293 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0d891a3-b1ac-468b-b4de-bf3faa5ad570-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-zzvzx\" (UID: \"a0d891a3-b1ac-468b-b4de-bf3faa5ad570\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zzvzx" Jul 22 02:37:44.147835 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:44.147803 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpw57\" (UniqueName: \"kubernetes.io/projected/a0d891a3-b1ac-468b-b4de-bf3faa5ad570-kube-api-access-xpw57\") pod \"payload-pre-processing-7677b774bb-zzvzx\" (UID: \"a0d891a3-b1ac-468b-b4de-bf3faa5ad570\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zzvzx" Jul 22 02:37:44.261510 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:44.261412 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-zzvzx" Jul 22 02:37:44.414491 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:44.414357 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-zzvzx"] Jul 22 02:37:44.417699 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:37:44.417664 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0d891a3_b1ac_468b_b4de_bf3faa5ad570.slice/crio-c0f30f370480c0f87737ded86d8dcb6c1de0b47f03113e5d6fa3d2f278403f49 WatchSource:0}: Error finding container c0f30f370480c0f87737ded86d8dcb6c1de0b47f03113e5d6fa3d2f278403f49: Status 404 returned error can't find the container with id c0f30f370480c0f87737ded86d8dcb6c1de0b47f03113e5d6fa3d2f278403f49 Jul 22 02:37:44.769064 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:44.769019 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-zzvzx" event={"ID":"a0d891a3-b1ac-468b-b4de-bf3faa5ad570","Type":"ContainerStarted","Data":"c0f30f370480c0f87737ded86d8dcb6c1de0b47f03113e5d6fa3d2f278403f49"} Jul 22 02:37:47.803372 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:47.803275 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-zzvzx" event={"ID":"a0d891a3-b1ac-468b-b4de-bf3faa5ad570","Type":"ContainerStarted","Data":"278e34b8b6c9fab419b09f0ab941c79e7131e6d2641e3c122b01197af058faaa"} Jul 22 02:37:47.803799 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:47.803442 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-zzvzx" Jul 22 02:37:47.820870 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:47.820812 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-zzvzx" podStartSLOduration=1.865545729 podStartE2EDuration="4.82079341s" totalImagesPullingTime="2.955247681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:37:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:37:47.818813839 +0000 UTC m=+592.667670673" watchObservedRunningTime="2026-07-22 02:37:47.82079341 +0000 UTC m=+592.669650287" Jul 22 02:37:51.312215 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:51.312157 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-flz22 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 02:37:51.312636 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:51.312237 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" podUID="53ddb162-08b8-45cd-b2ed-d3557318eb6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 02:37:51.827712 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:51.827679 2579 generic.go:356] "Generic (PLEG): container finished" podID="53ddb162-08b8-45cd-b2ed-d3557318eb6f" containerID="e93fcf983d241d8a1b4d8967bd7ef128fef4b3f47bc029f1d8be0835c28311f9" exitCode=0 Jul 22 02:37:51.827923 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:51.827747 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" event={"ID":"53ddb162-08b8-45cd-b2ed-d3557318eb6f","Type":"ContainerDied","Data":"e93fcf983d241d8a1b4d8967bd7ef128fef4b3f47bc029f1d8be0835c28311f9"} Jul 22 02:37:51.828183 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:51.828169 2579 scope.go:117] "RemoveContainer" containerID="e93fcf983d241d8a1b4d8967bd7ef128fef4b3f47bc029f1d8be0835c28311f9" Jul 22 02:37:51.829206 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:51.829178 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:37:51.829287 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:51.829248 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:37:51.829287 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:51.829281 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:37:52.835659 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:52.835620 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" event={"ID":"53ddb162-08b8-45cd-b2ed-d3557318eb6f","Type":"ContainerStarted","Data":"4ee8abb314dd32287220ec2b6531cc7128d832dfc52673ef2e0c80c88ae1b04c"} Jul 22 02:37:52.887997 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:52.885630 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:37:52.887997 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:52.885773 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-flz22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 22 02:37:52.887997 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:52.885834 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" podUID="53ddb162-08b8-45cd-b2ed-d3557318eb6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 22 02:37:53.880704 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:53.880667 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-flz22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 22 02:37:53.881329 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:53.881295 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" podUID="53ddb162-08b8-45cd-b2ed-d3557318eb6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 22 02:37:54.283749 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:54.283634 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:54.378180 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:54.378141 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:37:54.881215 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:54.881171 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-flz22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 22 02:37:54.881855 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:54.881820 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" podUID="53ddb162-08b8-45cd-b2ed-d3557318eb6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 22 02:37:55.577004 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:55.576972 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 02:37:55.585967 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:55.584680 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 02:37:55.595299 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:55.595267 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 02:37:55.596646 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:55.596625 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 02:37:55.883409 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:55.881380 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-flz22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 22 02:37:55.883409 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:55.881451 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" podUID="53ddb162-08b8-45cd-b2ed-d3557318eb6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 22 02:37:56.881518 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:56.881477 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-flz22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 22 02:37:56.881755 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:56.881560 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" podUID="53ddb162-08b8-45cd-b2ed-d3557318eb6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 22 02:37:57.884859 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:57.884818 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:37:57.885297 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:57.885221 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:37:57.885963 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:57.885927 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flz22" Jul 22 02:37:58.810880 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:37:58.810852 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-zzvzx" Jul 22 02:38:08.704399 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.704368 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl"] Jul 22 02:38:08.709357 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.709331 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.712281 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.712255 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 02:38:08.713351 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.713329 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 22 02:38:08.713489 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.713373 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 02:38:08.713616 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.713601 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pj5rv\"" Jul 22 02:38:08.720892 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.720868 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl"] Jul 22 02:38:08.739897 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.739864 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:38:08.790736 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.790701 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.790925 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.790764 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.790925 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.790787 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.790925 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.790852 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.790925 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.790915 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2kc2\" (UniqueName: \"kubernetes.io/projected/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-kube-api-access-v2kc2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.791099 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.791020 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.891731 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.891692 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.891731 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.891732 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.892027 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.891750 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.892027 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.891783 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2kc2\" (UniqueName: \"kubernetes.io/projected/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-kube-api-access-v2kc2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.892027 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.891831 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.892027 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.891866 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.892279 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.892209 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.892279 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.892239 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.892279 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.892272 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.894353 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.894329 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.894501 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.894486 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:08.901065 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:08.901037 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2kc2\" (UniqueName: \"kubernetes.io/projected/aa51ef7e-691c-4919-92dc-e0eb24ed4b77-kube-api-access-v2kc2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl\" (UID: \"aa51ef7e-691c-4919-92dc-e0eb24ed4b77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:09.020540 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:09.020443 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:09.173069 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:09.173037 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl"] Jul 22 02:38:09.175375 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:38:09.175343 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa51ef7e_691c_4919_92dc_e0eb24ed4b77.slice/crio-258189b0e7b9a758fda5a8f19f9d747fc4149cd8fa85d45a5c739442178d9a00 WatchSource:0}: Error finding container 258189b0e7b9a758fda5a8f19f9d747fc4149cd8fa85d45a5c739442178d9a00: Status 404 returned error can't find the container with id 258189b0e7b9a758fda5a8f19f9d747fc4149cd8fa85d45a5c739442178d9a00 Jul 22 02:38:09.928524 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:09.928491 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" event={"ID":"aa51ef7e-691c-4919-92dc-e0eb24ed4b77","Type":"ContainerStarted","Data":"258189b0e7b9a758fda5a8f19f9d747fc4149cd8fa85d45a5c739442178d9a00"} Jul 22 02:38:10.936202 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:10.936165 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" event={"ID":"aa51ef7e-691c-4919-92dc-e0eb24ed4b77","Type":"ContainerStarted","Data":"2789e5fbef1b43402450a7efaeb3b1865e10fa99ce73d94e044a03cc37fc0028"} Jul 22 02:38:10.962805 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:10.962750 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" podStartSLOduration=1.826234573 podStartE2EDuration="2.962728308s" totalImagesPullingTime="1.136493735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:38:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:38:10.962465079 +0000 UTC m=+615.811321912" watchObservedRunningTime="2026-07-22 02:38:10.962728308 +0000 UTC m=+615.811585140" Jul 22 02:38:19.020738 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:19.020638 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:19.020738 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:19.020708 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:19.034269 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:19.034240 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:19.999509 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:19.999471 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl" Jul 22 02:38:23.093479 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.093432 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dn244"] Jul 22 02:38:23.098884 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.098850 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.101491 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.101468 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 22 02:38:23.107889 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.107857 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dn244"] Jul 22 02:38:23.125668 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.125625 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:38:23.229452 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.229419 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60327b17-f4d5-4837-b9df-27c4f7b16ba5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.229452 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.229466 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60327b17-f4d5-4837-b9df-27c4f7b16ba5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.229688 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.229489 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5z4j\" (UniqueName: \"kubernetes.io/projected/60327b17-f4d5-4837-b9df-27c4f7b16ba5-kube-api-access-k5z4j\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.229688 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.229586 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60327b17-f4d5-4837-b9df-27c4f7b16ba5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.229688 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.229632 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60327b17-f4d5-4837-b9df-27c4f7b16ba5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.229688 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.229671 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60327b17-f4d5-4837-b9df-27c4f7b16ba5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.330690 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.330648 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60327b17-f4d5-4837-b9df-27c4f7b16ba5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.330690 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.330691 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5z4j\" (UniqueName: \"kubernetes.io/projected/60327b17-f4d5-4837-b9df-27c4f7b16ba5-kube-api-access-k5z4j\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.330937 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.330725 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60327b17-f4d5-4837-b9df-27c4f7b16ba5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.330937 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.330757 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60327b17-f4d5-4837-b9df-27c4f7b16ba5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.330937 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.330789 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60327b17-f4d5-4837-b9df-27c4f7b16ba5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.330937 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.330833 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60327b17-f4d5-4837-b9df-27c4f7b16ba5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.331198 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.331178 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60327b17-f4d5-4837-b9df-27c4f7b16ba5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.331258 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.331193 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60327b17-f4d5-4837-b9df-27c4f7b16ba5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.331358 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.331311 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60327b17-f4d5-4837-b9df-27c4f7b16ba5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.333476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.333453 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60327b17-f4d5-4837-b9df-27c4f7b16ba5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.333720 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.333702 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60327b17-f4d5-4837-b9df-27c4f7b16ba5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.339091 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.339068 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5z4j\" (UniqueName: \"kubernetes.io/projected/60327b17-f4d5-4837-b9df-27c4f7b16ba5-kube-api-access-k5z4j\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dn244\" (UID: \"60327b17-f4d5-4837-b9df-27c4f7b16ba5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.414176 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.414088 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:23.569323 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.569298 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dn244"] Jul 22 02:38:23.572266 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:38:23.572236 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60327b17_f4d5_4837_b9df_27c4f7b16ba5.slice/crio-afadb456442f4dbf3f0c10924767ef24930ed6b32ddbc0501ed8779fd6863898 WatchSource:0}: Error finding container afadb456442f4dbf3f0c10924767ef24930ed6b32ddbc0501ed8779fd6863898: Status 404 returned error can't find the container with id afadb456442f4dbf3f0c10924767ef24930ed6b32ddbc0501ed8779fd6863898 Jul 22 02:38:23.573661 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:23.573640 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:38:24.005692 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:24.005590 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" event={"ID":"60327b17-f4d5-4837-b9df-27c4f7b16ba5","Type":"ContainerStarted","Data":"c9e920a595cc3bc45afcc396623440620416e53a44c4a56cd2bb7fa021465636"} Jul 22 02:38:24.005692 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:24.005656 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" event={"ID":"60327b17-f4d5-4837-b9df-27c4f7b16ba5","Type":"ContainerStarted","Data":"afadb456442f4dbf3f0c10924767ef24930ed6b32ddbc0501ed8779fd6863898"} Jul 22 02:38:24.026513 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:24.026463 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" podStartSLOduration=0.853843539 podStartE2EDuration="1.026444713s" totalImagesPullingTime="172.601174ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:38:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:38:24.02515452 +0000 UTC m=+628.874011355" watchObservedRunningTime="2026-07-22 02:38:24.026444713 +0000 UTC m=+628.875301547" Jul 22 02:38:33.414824 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:33.414778 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:33.415304 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:33.414883 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:33.428133 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:33.428102 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:34.077479 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:34.077448 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dn244" Jul 22 02:38:40.198770 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.198732 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq"] Jul 22 02:38:40.205148 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.205122 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.208065 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.208039 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 22 02:38:40.216781 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.216753 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq"] Jul 22 02:38:40.263623 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.263589 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:38:40.278741 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.278709 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.278741 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.278751 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmt7z\" (UniqueName: \"kubernetes.io/projected/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-kube-api-access-zmt7z\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.279058 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.278901 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.279058 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.279001 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.279058 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.279031 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.279202 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.279059 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.379809 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.379769 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.379809 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.379815 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.380143 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.379837 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.380143 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.379858 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.380143 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.379900 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.380143 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.379934 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmt7z\" (UniqueName: \"kubernetes.io/projected/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-kube-api-access-zmt7z\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.380358 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.380303 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.380358 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.380336 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.380429 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.380353 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.382400 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.382366 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.382540 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.382521 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.390735 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.390705 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmt7z\" (UniqueName: \"kubernetes.io/projected/3bbcd8b7-6e16-442f-9e6c-69b365e18b85-kube-api-access-zmt7z\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4psmq\" (UID: \"3bbcd8b7-6e16-442f-9e6c-69b365e18b85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.517490 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.517395 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:40.671469 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:40.671441 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq"] Jul 22 02:38:40.673899 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:38:40.673863 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bbcd8b7_6e16_442f_9e6c_69b365e18b85.slice/crio-bb7fa65519703fa601ae9ccf1b1f4e024f980ac13f91968279ad9ef79b8dcd34 WatchSource:0}: Error finding container bb7fa65519703fa601ae9ccf1b1f4e024f980ac13f91968279ad9ef79b8dcd34: Status 404 returned error can't find the container with id bb7fa65519703fa601ae9ccf1b1f4e024f980ac13f91968279ad9ef79b8dcd34 Jul 22 02:38:41.105977 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:41.105850 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" event={"ID":"3bbcd8b7-6e16-442f-9e6c-69b365e18b85","Type":"ContainerStarted","Data":"270283832be1ef50d776ea29493627142f72be3670b6105fc684175668ced270"} Jul 22 02:38:41.105977 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:41.105913 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" event={"ID":"3bbcd8b7-6e16-442f-9e6c-69b365e18b85","Type":"ContainerStarted","Data":"bb7fa65519703fa601ae9ccf1b1f4e024f980ac13f91968279ad9ef79b8dcd34"} Jul 22 02:38:41.129204 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:41.129146 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" podStartSLOduration=0.957847934 podStartE2EDuration="1.129128862s" totalImagesPullingTime="171.280928ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:38:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:38:41.125983576 +0000 UTC m=+645.974840400" watchObservedRunningTime="2026-07-22 02:38:41.129128862 +0000 UTC m=+645.977985694" Jul 22 02:38:46.230296 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:46.230253 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:38:50.518272 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:50.518236 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:50.518665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:50.518306 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:50.531346 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:50.531319 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:51.177814 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:51.177775 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4psmq" Jul 22 02:38:53.387111 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:38:53.387075 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:39:11.946556 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:39:11.946518 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:39:19.434659 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:39:19.434598 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:40:21.503439 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:40:21.503337 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:40:42.778031 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:40:42.777933 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:40:42.868442 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:40:42.868402 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:40:44.083930 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:40:44.083893 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:40:44.171023 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:40:44.170987 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:41:30.820596 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:41:30.820514 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6765967b89-grpzj"] Jul 22 02:41:30.821182 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:41:30.820736 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6765967b89-grpzj" podUID="3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac" containerName="manager" containerID="cri-o://75a64182f35a7f5f496bc08a41ce800da3e3470d51ead5ae9e10c94d7d11c007" gracePeriod=30 Jul 22 02:41:31.043577 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:41:31.043542 2579 generic.go:356] "Generic (PLEG): container finished" podID="3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac" containerID="75a64182f35a7f5f496bc08a41ce800da3e3470d51ead5ae9e10c94d7d11c007" exitCode=0 Jul 22 02:41:31.043760 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:41:31.043627 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6765967b89-grpzj" event={"ID":"3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac","Type":"ContainerDied","Data":"75a64182f35a7f5f496bc08a41ce800da3e3470d51ead5ae9e10c94d7d11c007"} Jul 22 02:41:31.086422 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:41:31.086398 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6765967b89-grpzj" Jul 22 02:41:31.179568 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:41:31.179538 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q24cg\" (UniqueName: \"kubernetes.io/projected/3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac-kube-api-access-q24cg\") pod \"3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac\" (UID: \"3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac\") " Jul 22 02:41:31.179568 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:41:31.179573 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac-webhook-cert\") pod \"3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac\" (UID: \"3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac\") " Jul 22 02:41:31.188133 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:41:31.188055 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac" (UID: "3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:41:31.188530 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:41:31.188505 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac-kube-api-access-q24cg" (OuterVolumeSpecName: "kube-api-access-q24cg") pod "3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac" (UID: "3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac"). InnerVolumeSpecName "kube-api-access-q24cg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:41:31.280590 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:41:31.280550 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q24cg\" (UniqueName: \"kubernetes.io/projected/3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac-kube-api-access-q24cg\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:41:31.280590 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:41:31.280588 2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac-webhook-cert\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:41:32.050337 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:41:32.050253 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6765967b89-grpzj" Jul 22 02:41:32.050784 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:41:32.050248 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6765967b89-grpzj" event={"ID":"3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac","Type":"ContainerDied","Data":"c0bc13661a968b57770d45ca12e836151096152c59bad4f0b2692083f3c98e70"} Jul 22 02:41:32.050784 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:41:32.050383 2579 scope.go:117] "RemoveContainer" containerID="75a64182f35a7f5f496bc08a41ce800da3e3470d51ead5ae9e10c94d7d11c007" Jul 22 02:41:32.067217 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:41:32.067187 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6765967b89-grpzj"] Jul 22 02:41:32.070420 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:41:32.070394 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6765967b89-grpzj"] Jul 22 02:41:33.628350 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:41:33.628311 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac" path="/var/lib/kubelet/pods/3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac/volumes" Jul 22 02:42:05.865602 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:05.865565 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:42:32.921187 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:32.921148 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6765967b89-wgwqk"] Jul 22 02:42:32.921576 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:32.921563 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac" containerName="manager" Jul 22 02:42:32.921619 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:32.921578 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac" containerName="manager" Jul 22 02:42:32.921662 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:32.921649 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cf0ed33-91f5-4ac6-a667-b08fe8bb03ac" containerName="manager" Jul 22 02:42:32.924809 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:32.924793 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6765967b89-wgwqk" Jul 22 02:42:32.928513 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:32.928490 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 22 02:42:32.928513 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:32.928506 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 22 02:42:32.928702 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:32.928490 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tkqdg\"" Jul 22 02:42:32.934957 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:32.934914 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6765967b89-wgwqk"] Jul 22 02:42:33.051476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:33.051436 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf-webhook-cert\") pod \"maas-controller-6765967b89-wgwqk\" (UID: \"f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf\") " pod="opendatahub/maas-controller-6765967b89-wgwqk" Jul 22 02:42:33.051476 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:33.051483 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlq6x\" (UniqueName: \"kubernetes.io/projected/f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf-kube-api-access-vlq6x\") pod \"maas-controller-6765967b89-wgwqk\" (UID: \"f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf\") " pod="opendatahub/maas-controller-6765967b89-wgwqk" Jul 22 02:42:33.152365 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:33.152326 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf-webhook-cert\") pod \"maas-controller-6765967b89-wgwqk\" (UID: \"f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf\") " pod="opendatahub/maas-controller-6765967b89-wgwqk" Jul 22 02:42:33.152365 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:33.152366 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlq6x\" (UniqueName: \"kubernetes.io/projected/f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf-kube-api-access-vlq6x\") pod \"maas-controller-6765967b89-wgwqk\" (UID: \"f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf\") " pod="opendatahub/maas-controller-6765967b89-wgwqk" Jul 22 02:42:33.155042 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:33.155018 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf-webhook-cert\") pod \"maas-controller-6765967b89-wgwqk\" (UID: \"f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf\") " pod="opendatahub/maas-controller-6765967b89-wgwqk" Jul 22 02:42:33.163285 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:33.163262 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlq6x\" (UniqueName: \"kubernetes.io/projected/f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf-kube-api-access-vlq6x\") pod \"maas-controller-6765967b89-wgwqk\" (UID: \"f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf\") " pod="opendatahub/maas-controller-6765967b89-wgwqk" Jul 22 02:42:33.237247 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:33.237156 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6765967b89-wgwqk" Jul 22 02:42:33.384981 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:33.384934 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6765967b89-wgwqk"] Jul 22 02:42:33.386642 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:42:33.386618 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5b7f6c4_cc69_44d9_bfcf_513fd9b479bf.slice/crio-9f2d1ad896f18280210c69491fe9d1afeaed72e8886847fc3627fee0fb1a8873 WatchSource:0}: Error finding container 9f2d1ad896f18280210c69491fe9d1afeaed72e8886847fc3627fee0fb1a8873: Status 404 returned error can't find the container with id 9f2d1ad896f18280210c69491fe9d1afeaed72e8886847fc3627fee0fb1a8873 Jul 22 02:42:34.395443 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:34.395403 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6765967b89-wgwqk" event={"ID":"f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf","Type":"ContainerStarted","Data":"05e5db1fb4fbfe0637000f13a820399f2feb2483c3f97624989d6544ba7e8ade"} Jul 22 02:42:34.395936 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:34.395458 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6765967b89-wgwqk" event={"ID":"f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf","Type":"ContainerStarted","Data":"9f2d1ad896f18280210c69491fe9d1afeaed72e8886847fc3627fee0fb1a8873"} Jul 22 02:42:34.395936 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:34.395539 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6765967b89-wgwqk" Jul 22 02:42:34.415598 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:34.415548 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6765967b89-wgwqk" podStartSLOduration=2.07262288 podStartE2EDuration="2.415530943s" totalImagesPullingTime="342.908063ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:42:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:42:34.413288779 +0000 UTC m=+879.262145611" watchObservedRunningTime="2026-07-22 02:42:34.415530943 +0000 UTC m=+879.264387776" Jul 22 02:42:45.403431 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:45.403385 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6765967b89-wgwqk" Jul 22 02:42:50.042322 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.042228 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:42:50.145635 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.145599 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6765967b89-wgwqk"] Jul 22 02:42:50.145916 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.145845 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6765967b89-wgwqk" podUID="f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf" containerName="manager" containerID="cri-o://05e5db1fb4fbfe0637000f13a820399f2feb2483c3f97624989d6544ba7e8ade" gracePeriod=30 Jul 22 02:42:50.406658 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.406629 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6765967b89-wgwqk" Jul 22 02:42:50.480885 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.480855 2579 generic.go:356] "Generic (PLEG): container finished" podID="f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf" containerID="05e5db1fb4fbfe0637000f13a820399f2feb2483c3f97624989d6544ba7e8ade" exitCode=0 Jul 22 02:42:50.481077 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.480901 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6765967b89-wgwqk" Jul 22 02:42:50.481077 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.480970 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6765967b89-wgwqk" event={"ID":"f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf","Type":"ContainerDied","Data":"05e5db1fb4fbfe0637000f13a820399f2feb2483c3f97624989d6544ba7e8ade"} Jul 22 02:42:50.481077 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.481027 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6765967b89-wgwqk" event={"ID":"f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf","Type":"ContainerDied","Data":"9f2d1ad896f18280210c69491fe9d1afeaed72e8886847fc3627fee0fb1a8873"} Jul 22 02:42:50.481077 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.481056 2579 scope.go:117] "RemoveContainer" containerID="05e5db1fb4fbfe0637000f13a820399f2feb2483c3f97624989d6544ba7e8ade" Jul 22 02:42:50.490456 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.490435 2579 scope.go:117] "RemoveContainer" containerID="05e5db1fb4fbfe0637000f13a820399f2feb2483c3f97624989d6544ba7e8ade" Jul 22 02:42:50.490771 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:42:50.490746 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05e5db1fb4fbfe0637000f13a820399f2feb2483c3f97624989d6544ba7e8ade\": container with ID starting with 05e5db1fb4fbfe0637000f13a820399f2feb2483c3f97624989d6544ba7e8ade not found: ID does not exist" containerID="05e5db1fb4fbfe0637000f13a820399f2feb2483c3f97624989d6544ba7e8ade" Jul 22 02:42:50.490888 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.490785 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e5db1fb4fbfe0637000f13a820399f2feb2483c3f97624989d6544ba7e8ade"} err="failed to get container status \"05e5db1fb4fbfe0637000f13a820399f2feb2483c3f97624989d6544ba7e8ade\": rpc error: code = NotFound desc = could not find container \"05e5db1fb4fbfe0637000f13a820399f2feb2483c3f97624989d6544ba7e8ade\": container with ID starting with 05e5db1fb4fbfe0637000f13a820399f2feb2483c3f97624989d6544ba7e8ade not found: ID does not exist" Jul 22 02:42:50.505782 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.505754 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf-webhook-cert\") pod \"f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf\" (UID: \"f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf\") " Jul 22 02:42:50.505916 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.505879 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlq6x\" (UniqueName: \"kubernetes.io/projected/f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf-kube-api-access-vlq6x\") pod \"f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf\" (UID: \"f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf\") " Jul 22 02:42:50.508013 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.507978 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf" (UID: "f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:42:50.508126 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.508026 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf-kube-api-access-vlq6x" (OuterVolumeSpecName: "kube-api-access-vlq6x") pod "f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf" (UID: "f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf"). InnerVolumeSpecName "kube-api-access-vlq6x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:42:50.606629 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.606593 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlq6x\" (UniqueName: \"kubernetes.io/projected/f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf-kube-api-access-vlq6x\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:42:50.606629 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.606622 2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf-webhook-cert\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:42:50.803569 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.803537 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6765967b89-wgwqk"] Jul 22 02:42:50.808068 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:50.808035 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6765967b89-wgwqk"] Jul 22 02:42:51.581553 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:51.581522 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:42:51.629377 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:51.629341 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf" path="/var/lib/kubelet/pods/f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf/volumes" Jul 22 02:42:55.623515 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:55.623478 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 02:42:55.626053 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:55.624221 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 02:42:55.631356 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:55.631331 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 02:42:55.632176 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:42:55.632155 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 02:43:30.250842 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:43:30.250795 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:43:30.346003 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:43:30.345930 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:43:30.563164 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:43:30.563043 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:43:36.765608 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:43:36.765551 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:43:36.831393 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:43:36.831347 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:43:36.930882 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:43:36.930842 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:43:37.044093 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:43:37.044005 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:43:37.185273 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:43:37.185238 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:43:43.004843 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:43:43.004810 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:43:43.076235 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:43:43.076202 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:45:00.155013 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:00.154973 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744805-55t8g"] Jul 22 02:45:00.155516 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:00.155435 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf" containerName="manager" Jul 22 02:45:00.155516 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:00.155451 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf" containerName="manager" Jul 22 02:45:00.155603 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:00.155524 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5b7f6c4-cc69-44d9-bfcf-513fd9b479bf" containerName="manager" Jul 22 02:45:00.158991 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:00.158969 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744805-55t8g" Jul 22 02:45:00.162091 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:00.162064 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tm89v\"" Jul 22 02:45:00.169897 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:00.169871 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744805-55t8g" Jul 22 02:45:00.177684 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:00.177650 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744805-55t8g"] Jul 22 02:45:00.540281 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:00.540255 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744805-55t8g"] Jul 22 02:45:00.542317 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:45:00.542287 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02f5ca2b_6521_47e5_b53b_9d3436fcb2af.slice/crio-74f5da7d7ed4fd8a29bb27542abad141f41d84c78ef503e6f6b4e168a08e29e1 WatchSource:0}: Error finding container 74f5da7d7ed4fd8a29bb27542abad141f41d84c78ef503e6f6b4e168a08e29e1: Status 404 returned error can't find the container with id 74f5da7d7ed4fd8a29bb27542abad141f41d84c78ef503e6f6b4e168a08e29e1 Jul 22 02:45:00.544279 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:00.544263 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:45:01.191843 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:01.191803 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744805-55t8g" event={"ID":"02f5ca2b-6521-47e5-b53b-9d3436fcb2af","Type":"ContainerStarted","Data":"74f5da7d7ed4fd8a29bb27542abad141f41d84c78ef503e6f6b4e168a08e29e1"} Jul 22 02:45:01.888896 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:01.888856 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:45:04.211392 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:04.211346 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744805-55t8g" event={"ID":"02f5ca2b-6521-47e5-b53b-9d3436fcb2af","Type":"ContainerStarted","Data":"9cf73063a732840af823897855cafc86e9aca164b721a267d4f38b52b66210a3"} Jul 22 02:45:04.228013 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:04.227930 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744805-55t8g" podStartSLOduration=1.373824767 podStartE2EDuration="4.227911186s" totalImagesPullingTime="2.854086419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:45:04.225986471 +0000 UTC m=+1029.074843298" watchObservedRunningTime="2026-07-22 02:45:04.227911186 +0000 UTC m=+1029.076768019" Jul 22 02:45:05.218418 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:05.218378 2579 generic.go:356] "Generic (PLEG): container finished" podID="02f5ca2b-6521-47e5-b53b-9d3436fcb2af" containerID="9cf73063a732840af823897855cafc86e9aca164b721a267d4f38b52b66210a3" exitCode=0 Jul 22 02:45:05.218812 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:05.218464 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744805-55t8g" event={"ID":"02f5ca2b-6521-47e5-b53b-9d3436fcb2af","Type":"ContainerDied","Data":"9cf73063a732840af823897855cafc86e9aca164b721a267d4f38b52b66210a3"} Jul 22 02:45:06.381037 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:06.381010 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744805-55t8g" Jul 22 02:45:07.231454 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:07.231408 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744805-55t8g" event={"ID":"02f5ca2b-6521-47e5-b53b-9d3436fcb2af","Type":"ContainerDied","Data":"74f5da7d7ed4fd8a29bb27542abad141f41d84c78ef503e6f6b4e168a08e29e1"} Jul 22 02:45:07.231741 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:07.231475 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="74f5da7d7ed4fd8a29bb27542abad141f41d84c78ef503e6f6b4e168a08e29e1" Jul 22 02:45:07.231741 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:45:07.231424 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744805-55t8g" Jul 22 02:46:23.132637 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:46:23.132554 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:46:23.233503 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:46:23.233460 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:46:32.572248 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:46:32.572206 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:46:32.673277 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:46:32.673240 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:47:40.158549 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:40.158468 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw"] Jul 22 02:47:40.159159 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:40.158691 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" podUID="e77582da-db54-4cab-b409-62086772faad" containerName="manager" containerID="cri-o://f423a3408368f3c3d008826828df346ff879ea29baa61e7026eb005d7f8f3c16" gracePeriod=10 Jul 22 02:47:42.727360 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:42.727334 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" Jul 22 02:47:42.859254 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:42.859220 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcd4z\" (UniqueName: \"kubernetes.io/projected/e77582da-db54-4cab-b409-62086772faad-kube-api-access-qcd4z\") pod \"e77582da-db54-4cab-b409-62086772faad\" (UID: \"e77582da-db54-4cab-b409-62086772faad\") " Jul 22 02:47:42.859471 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:42.859281 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e77582da-db54-4cab-b409-62086772faad-extensions-socket-volume\") pod \"e77582da-db54-4cab-b409-62086772faad\" (UID: \"e77582da-db54-4cab-b409-62086772faad\") " Jul 22 02:47:42.859734 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:42.859707 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e77582da-db54-4cab-b409-62086772faad-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e77582da-db54-4cab-b409-62086772faad" (UID: "e77582da-db54-4cab-b409-62086772faad"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:47:42.861702 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:42.861668 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e77582da-db54-4cab-b409-62086772faad-kube-api-access-qcd4z" (OuterVolumeSpecName: "kube-api-access-qcd4z") pod "e77582da-db54-4cab-b409-62086772faad" (UID: "e77582da-db54-4cab-b409-62086772faad"). InnerVolumeSpecName "kube-api-access-qcd4z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:47:42.960657 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:42.960613 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcd4z\" (UniqueName: \"kubernetes.io/projected/e77582da-db54-4cab-b409-62086772faad-kube-api-access-qcd4z\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:47:42.960657 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:42.960650 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e77582da-db54-4cab-b409-62086772faad-extensions-socket-volume\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:47:43.096879 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:43.096839 2579 generic.go:356] "Generic (PLEG): container finished" podID="e77582da-db54-4cab-b409-62086772faad" containerID="f423a3408368f3c3d008826828df346ff879ea29baa61e7026eb005d7f8f3c16" exitCode=0 Jul 22 02:47:43.097109 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:43.096895 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" Jul 22 02:47:43.097109 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:43.096920 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" event={"ID":"e77582da-db54-4cab-b409-62086772faad","Type":"ContainerDied","Data":"f423a3408368f3c3d008826828df346ff879ea29baa61e7026eb005d7f8f3c16"} Jul 22 02:47:43.097109 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:43.097016 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw" event={"ID":"e77582da-db54-4cab-b409-62086772faad","Type":"ContainerDied","Data":"2e541732b0a5729462a8ae7ccf3fd694a63da77eaff8624ef6f305d5a22871e1"} Jul 22 02:47:43.097109 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:43.097043 2579 scope.go:117] "RemoveContainer" containerID="f423a3408368f3c3d008826828df346ff879ea29baa61e7026eb005d7f8f3c16" Jul 22 02:47:43.108394 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:43.108369 2579 scope.go:117] "RemoveContainer" containerID="f423a3408368f3c3d008826828df346ff879ea29baa61e7026eb005d7f8f3c16" Jul 22 02:47:43.108688 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:47:43.108664 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f423a3408368f3c3d008826828df346ff879ea29baa61e7026eb005d7f8f3c16\": container with ID starting with f423a3408368f3c3d008826828df346ff879ea29baa61e7026eb005d7f8f3c16 not found: ID does not exist" containerID="f423a3408368f3c3d008826828df346ff879ea29baa61e7026eb005d7f8f3c16" Jul 22 02:47:43.108733 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:43.108698 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f423a3408368f3c3d008826828df346ff879ea29baa61e7026eb005d7f8f3c16"} err="failed to get container status \"f423a3408368f3c3d008826828df346ff879ea29baa61e7026eb005d7f8f3c16\": rpc error: code = NotFound desc = could not find container \"f423a3408368f3c3d008826828df346ff879ea29baa61e7026eb005d7f8f3c16\": container with ID starting with f423a3408368f3c3d008826828df346ff879ea29baa61e7026eb005d7f8f3c16 not found: ID does not exist" Jul 22 02:47:43.121687 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:43.121608 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw"] Jul 22 02:47:43.126786 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:43.126756 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n6ksw"] Jul 22 02:47:43.629338 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:43.629301 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e77582da-db54-4cab-b409-62086772faad" path="/var/lib/kubelet/pods/e77582da-db54-4cab-b409-62086772faad/volumes" Jul 22 02:47:55.667032 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:55.666886 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 02:47:55.678582 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:55.670340 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 02:47:55.678582 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:55.674220 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 02:47:55.678582 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:47:55.677576 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 02:48:09.131611 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:09.131570 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:48:41.401789 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.401752 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jrhd7"] Jul 22 02:48:41.402427 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.402365 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e77582da-db54-4cab-b409-62086772faad" containerName="manager" Jul 22 02:48:41.402427 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.402384 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e77582da-db54-4cab-b409-62086772faad" containerName="manager" Jul 22 02:48:41.402427 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.402400 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02f5ca2b-6521-47e5-b53b-9d3436fcb2af" containerName="cleanup" Jul 22 02:48:41.402427 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.402408 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="02f5ca2b-6521-47e5-b53b-9d3436fcb2af" containerName="cleanup" Jul 22 02:48:41.402661 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.402569 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e77582da-db54-4cab-b409-62086772faad" containerName="manager" Jul 22 02:48:41.402661 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.402582 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="02f5ca2b-6521-47e5-b53b-9d3436fcb2af" containerName="cleanup" Jul 22 02:48:41.405838 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.405812 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jrhd7" Jul 22 02:48:41.408664 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.408642 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d4w27\"" Jul 22 02:48:41.417918 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.417889 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jrhd7"] Jul 22 02:48:41.596742 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.596700 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ggxp\" (UniqueName: \"kubernetes.io/projected/89089245-cb77-4847-b496-fc24592a732b-kube-api-access-6ggxp\") pod \"kuadrant-operator-controller-manager-86878c769f-jrhd7\" (UID: \"89089245-cb77-4847-b496-fc24592a732b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jrhd7" Jul 22 02:48:41.596924 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.596830 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89089245-cb77-4847-b496-fc24592a732b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jrhd7\" (UID: \"89089245-cb77-4847-b496-fc24592a732b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jrhd7" Jul 22 02:48:41.697869 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.697771 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89089245-cb77-4847-b496-fc24592a732b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jrhd7\" (UID: \"89089245-cb77-4847-b496-fc24592a732b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jrhd7" Jul 22 02:48:41.697869 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.697848 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ggxp\" (UniqueName: \"kubernetes.io/projected/89089245-cb77-4847-b496-fc24592a732b-kube-api-access-6ggxp\") pod \"kuadrant-operator-controller-manager-86878c769f-jrhd7\" (UID: \"89089245-cb77-4847-b496-fc24592a732b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jrhd7" Jul 22 02:48:41.698230 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.698211 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89089245-cb77-4847-b496-fc24592a732b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jrhd7\" (UID: \"89089245-cb77-4847-b496-fc24592a732b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jrhd7" Jul 22 02:48:41.707783 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.707751 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ggxp\" (UniqueName: \"kubernetes.io/projected/89089245-cb77-4847-b496-fc24592a732b-kube-api-access-6ggxp\") pod \"kuadrant-operator-controller-manager-86878c769f-jrhd7\" (UID: \"89089245-cb77-4847-b496-fc24592a732b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jrhd7" Jul 22 02:48:41.717616 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.717587 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jrhd7" Jul 22 02:48:41.877116 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:41.877086 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jrhd7"] Jul 22 02:48:41.879014 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:48:41.878984 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89089245_cb77_4847_b496_fc24592a732b.slice/crio-adc7887c71df1130fd0490126d5851ae50db84c24392cbf4ce9abc94f31da702 WatchSource:0}: Error finding container adc7887c71df1130fd0490126d5851ae50db84c24392cbf4ce9abc94f31da702: Status 404 returned error can't find the container with id adc7887c71df1130fd0490126d5851ae50db84c24392cbf4ce9abc94f31da702 Jul 22 02:48:42.415686 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:42.415646 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jrhd7" event={"ID":"89089245-cb77-4847-b496-fc24592a732b","Type":"ContainerStarted","Data":"9c15f72eb80f528dff1106479ef626d563fde9a00460addc1bbd4ad3bdba9d05"} Jul 22 02:48:42.416168 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:42.415711 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jrhd7" event={"ID":"89089245-cb77-4847-b496-fc24592a732b","Type":"ContainerStarted","Data":"adc7887c71df1130fd0490126d5851ae50db84c24392cbf4ce9abc94f31da702"} Jul 22 02:48:42.416168 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:42.415739 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jrhd7" Jul 22 02:48:42.436973 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:42.436884 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jrhd7" podStartSLOduration=1.436867717 podStartE2EDuration="1.436867717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:48:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:48:42.435228023 +0000 UTC m=+1247.284084857" watchObservedRunningTime="2026-07-22 02:48:42.436867717 +0000 UTC m=+1247.285724624" Jul 22 02:48:53.423000 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:48:53.422971 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jrhd7" Jul 22 02:51:19.150049 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:51:19.150018 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:51:32.342555 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:51:32.342504 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:51:55.296223 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:51:55.296174 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:51:56.234006 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:51:56.233930 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:52:39.648926 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:52:39.648887 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:52:55.698785 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:52:55.698670 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 02:52:55.704060 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:52:55.704038 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 02:52:55.705795 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:52:55.705777 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 02:52:55.711215 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:52:55.711197 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 02:54:03.653627 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:54:03.653585 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:54:05.167729 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:54:05.167689 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:54:05.407725 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:54:05.407676 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:55:04.148444 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:55:04.148336 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:55:25.715513 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:55:25.715475 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:55:28.814639 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:55:28.814601 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:55:28.914456 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:55:28.914413 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:55:29.326523 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:55:29.326481 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:55:35.286294 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:55:35.286251 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:55:35.387858 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:55:35.387829 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:55:35.533868 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:55:35.533830 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:55:35.873274 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:55:35.873187 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:55:36.039294 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:55:36.039254 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:55:38.618132 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:55:38.618089 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:55:38.709160 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:55:38.709123 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:55:38.794851 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:55:38.794813 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:55:38.900773 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:55:38.900678 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:55:39.440175 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:55:39.440136 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:55:39.540254 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:55:39.540217 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:56:04.295566 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:04.295521 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns"] Jul 22 02:56:04.299046 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:04.299026 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns" Jul 22 02:56:04.302082 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:04.302054 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 02:56:04.302186 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:04.302071 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tm89v\"" Jul 22 02:56:04.308764 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:04.308731 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns"] Jul 22 02:56:04.366512 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:04.366470 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns\" (UID: \"e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns" Jul 22 02:56:04.467804 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:04.467765 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns\" (UID: \"e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns" Jul 22 02:56:04.468413 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:04.468394 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns\" (UID: \"e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns" Jul 22 02:56:04.611033 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:04.611000 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns" Jul 22 02:56:04.779370 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:04.779337 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns"] Jul 22 02:56:04.781240 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:56:04.781204 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6ca45c6_1edd_4f7c_9b53_0b415dd41dd0.slice/crio-6906e9d36e2d364687971041ddb0ff644dc2074a0fd2882ad58746d97ab71d29 WatchSource:0}: Error finding container 6906e9d36e2d364687971041ddb0ff644dc2074a0fd2882ad58746d97ab71d29: Status 404 returned error can't find the container with id 6906e9d36e2d364687971041ddb0ff644dc2074a0fd2882ad58746d97ab71d29 Jul 22 02:56:04.822038 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:04.822008 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns" event={"ID":"e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0","Type":"ContainerStarted","Data":"6906e9d36e2d364687971041ddb0ff644dc2074a0fd2882ad58746d97ab71d29"} Jul 22 02:56:05.828649 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:05.828607 2579 generic.go:356] "Generic (PLEG): container finished" podID="e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0" containerID="d632b3b3ad3b843052abbf20935efa4924efd07a8111f044bb9679420a23a5c9" exitCode=0 Jul 22 02:56:05.829057 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:05.828702 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns" event={"ID":"e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0","Type":"ContainerDied","Data":"d632b3b3ad3b843052abbf20935efa4924efd07a8111f044bb9679420a23a5c9"} Jul 22 02:56:06.978470 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:06.978447 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns" Jul 22 02:56:06.990593 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:06.990564 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0-maas-api-service-ca\") pod \"e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0\" (UID: \"e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0\") " Jul 22 02:56:06.990986 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:06.990928 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0" (UID: "e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:56:07.091564 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:07.091467 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0-maas-api-service-ca\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:56:07.840527 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:07.840494 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns" Jul 22 02:56:07.840527 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:07.840504 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns" event={"ID":"e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0","Type":"ContainerDied","Data":"6906e9d36e2d364687971041ddb0ff644dc2074a0fd2882ad58746d97ab71d29"} Jul 22 02:56:07.840740 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:07.840558 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6906e9d36e2d364687971041ddb0ff644dc2074a0fd2882ad58746d97ab71d29" Jul 22 02:56:08.466275 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:08.466238 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:56:08.566296 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:08.566260 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:56:14.804857 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:14.804818 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:56:19.375111 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:19.375024 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns"] Jul 22 02:56:19.376542 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:19.376490 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:56:19.377397 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:19.377375 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c6d7b74-m64ns"] Jul 22 02:56:19.542414 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:19.542370 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:56:19.629473 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:19.629381 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0" path="/var/lib/kubelet/pods/e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0/volumes" Jul 22 02:56:19.776065 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:19.776025 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:56:20.019896 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.019796 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:56:20.119923 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.119883 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:56:20.255913 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.255879 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:56:20.560024 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.559989 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v"] Jul 22 02:56:20.560598 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.560577 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0" containerName="revoke-keys" Jul 22 02:56:20.560690 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.560602 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0" containerName="revoke-keys" Jul 22 02:56:20.560743 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.560720 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6ca45c6-1edd-4f7c-9b53-0b415dd41dd0" containerName="revoke-keys" Jul 22 02:56:20.564527 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.564496 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.567004 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.566934 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-113f7429-gw-openshift-default-dockercfg-n98gv\"" Jul 22 02:56:20.582168 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.582068 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v"] Jul 22 02:56:20.588501 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.588470 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:56:20.622848 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.622816 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-workload-socket\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.622848 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.622852 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-workload-certs\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.623141 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.622930 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ee9d700-7952-4277-937c-96af910a747d-istiod-ca-cert\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.623141 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.623000 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-istio-envoy\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.623141 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.623042 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ee9d700-7952-4277-937c-96af910a747d-istio-token\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.623141 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.623075 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsrq6\" (UniqueName: \"kubernetes.io/projected/9ee9d700-7952-4277-937c-96af910a747d-kube-api-access-tsrq6\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.623337 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.623142 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-credential-socket\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.623337 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.623167 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-istio-data\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.623337 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.623195 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ee9d700-7952-4277-937c-96af910a747d-istio-podinfo\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.703856 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.703807 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:56:20.724699 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.724655 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-credential-socket\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.724874 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.724699 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-istio-data\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.724874 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.724813 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ee9d700-7952-4277-937c-96af910a747d-istio-podinfo\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.725041 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.724892 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-workload-socket\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.725041 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.724930 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-workload-certs\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.725154 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.725057 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ee9d700-7952-4277-937c-96af910a747d-istiod-ca-cert\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.725154 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.725109 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-istio-envoy\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.725263 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.725157 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-credential-socket\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.725263 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.725160 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ee9d700-7952-4277-937c-96af910a747d-istio-token\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.725263 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.725185 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-istio-data\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.725418 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.725260 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsrq6\" (UniqueName: \"kubernetes.io/projected/9ee9d700-7952-4277-937c-96af910a747d-kube-api-access-tsrq6\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.725418 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.725303 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-workload-socket\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.725716 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.725688 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-workload-certs\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.725823 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.725778 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ee9d700-7952-4277-937c-96af910a747d-istiod-ca-cert\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.727664 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.727640 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ee9d700-7952-4277-937c-96af910a747d-istio-podinfo\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.727754 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.727640 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-istio-envoy\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.733543 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.733519 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ee9d700-7952-4277-937c-96af910a747d-istio-token\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.733748 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.733731 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsrq6\" (UniqueName: \"kubernetes.io/projected/9ee9d700-7952-4277-937c-96af910a747d-kube-api-access-tsrq6\") pod \"e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:20.881937 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:20.881898 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:21.036818 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:21.036784 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v"] Jul 22 02:56:21.039346 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:56:21.039317 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ee9d700_7952_4277_937c_96af910a747d.slice/crio-4ba2947f2d95e9ef8526b67212d77083c35351be2d00d692a2ae58773d61fa19 WatchSource:0}: Error finding container 4ba2947f2d95e9ef8526b67212d77083c35351be2d00d692a2ae58773d61fa19: Status 404 returned error can't find the container with id 4ba2947f2d95e9ef8526b67212d77083c35351be2d00d692a2ae58773d61fa19 Jul 22 02:56:21.041977 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:21.041918 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:56:21.042075 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:21.042015 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:56:21.042075 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:21.042049 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:56:21.924512 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:21.924470 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" event={"ID":"9ee9d700-7952-4277-937c-96af910a747d","Type":"ContainerStarted","Data":"bef123d1373f075656ecde22741eced6072b1b8e88ef3b697d598b8bf6cc40bf"} Jul 22 02:56:21.924933 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:21.924573 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" event={"ID":"9ee9d700-7952-4277-937c-96af910a747d","Type":"ContainerStarted","Data":"4ba2947f2d95e9ef8526b67212d77083c35351be2d00d692a2ae58773d61fa19"} Jul 22 02:56:21.945635 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:21.945558 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" podStartSLOduration=1.945540458 podStartE2EDuration="1.945540458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:56:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:56:21.944579727 +0000 UTC m=+1706.793436560" watchObservedRunningTime="2026-07-22 02:56:21.945540458 +0000 UTC m=+1706.794397291" Jul 22 02:56:22.883124 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:22.883084 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:22.888356 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:22.888328 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:22.930206 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:22.930179 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:22.931126 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:22.931102 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:23.407432 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:23.407397 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:56:23.582296 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:23.582261 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:56:23.700366 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:23.700269 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:56:24.137356 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:24.137317 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:56:24.238282 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:24.238249 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:56:37.932201 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:37.932107 2579 patch_prober.go:28] interesting pod/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 22 02:56:37.932656 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:37.932208 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" podUID="9ee9d700-7952-4277-937c-96af910a747d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 22 02:56:38.023004 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:38.022971 2579 generic.go:356] "Generic (PLEG): container finished" podID="9ee9d700-7952-4277-937c-96af910a747d" containerID="bef123d1373f075656ecde22741eced6072b1b8e88ef3b697d598b8bf6cc40bf" exitCode=0 Jul 22 02:56:38.023216 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:38.023039 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" event={"ID":"9ee9d700-7952-4277-937c-96af910a747d","Type":"ContainerDied","Data":"bef123d1373f075656ecde22741eced6072b1b8e88ef3b697d598b8bf6cc40bf"} Jul 22 02:56:38.023529 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:38.023511 2579 scope.go:117] "RemoveContainer" containerID="bef123d1373f075656ecde22741eced6072b1b8e88ef3b697d598b8bf6cc40bf" Jul 22 02:56:38.024726 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:38.024696 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:56:38.024805 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:38.024788 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:56:38.024847 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:38.024833 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:56:39.030198 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:39.030157 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" event={"ID":"9ee9d700-7952-4277-937c-96af910a747d","Type":"ContainerStarted","Data":"9af52cc2caa5e17b6624e94b3ddc47466701f0e34b788f34ffb181f350ec9f19"} Jul 22 02:56:39.882678 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:39.882637 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:39.882899 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:39.882771 2579 patch_prober.go:28] interesting pod/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 22 02:56:39.882899 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:39.882819 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" podUID="9ee9d700-7952-4277-937c-96af910a747d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 22 02:56:40.883448 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:40.883382 2579 patch_prober.go:28] interesting pod/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 22 02:56:40.884010 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:40.883472 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" podUID="9ee9d700-7952-4277-937c-96af910a747d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 22 02:56:41.883060 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:41.883015 2579 patch_prober.go:28] interesting pod/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 22 02:56:41.883281 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:41.883084 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" podUID="9ee9d700-7952-4277-937c-96af910a747d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 22 02:56:42.882934 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:42.882883 2579 patch_prober.go:28] interesting pod/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 22 02:56:42.883402 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:42.882975 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" podUID="9ee9d700-7952-4277-937c-96af910a747d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 22 02:56:43.886522 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:43.886492 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:43.886936 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:43.886887 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:43.887654 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:43.887631 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:56:58.948069 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:56:58.948010 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:03.524812 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:03.524771 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:03.672002 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:03.671963 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v"] Jul 22 02:57:03.672282 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:03.672253 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" podUID="9ee9d700-7952-4277-937c-96af910a747d" containerName="istio-proxy" containerID="cri-o://9af52cc2caa5e17b6624e94b3ddc47466701f0e34b788f34ffb181f350ec9f19" gracePeriod=30 Jul 22 02:57:03.686601 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:03.686561 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:03.932622 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:03.932581 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:04.168331 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:04.168289 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:04.520323 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:04.520287 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:04.589186 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:04.589144 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:07.073810 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:07.073766 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:07.159957 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:07.159906 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:07.233676 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:07.233635 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:07.401037 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:07.400997 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:07.752816 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:07.752726 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:07.822157 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:07.822119 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:08.956158 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:08.956131 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:57:09.088767 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.088722 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-workload-certs\") pod \"9ee9d700-7952-4277-937c-96af910a747d\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " Jul 22 02:57:09.088767 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.088770 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ee9d700-7952-4277-937c-96af910a747d-istiod-ca-cert\") pod \"9ee9d700-7952-4277-937c-96af910a747d\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " Jul 22 02:57:09.089085 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.088797 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ee9d700-7952-4277-937c-96af910a747d-istio-podinfo\") pod \"9ee9d700-7952-4277-937c-96af910a747d\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " Jul 22 02:57:09.089085 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.088833 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-istio-envoy\") pod \"9ee9d700-7952-4277-937c-96af910a747d\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " Jul 22 02:57:09.089085 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.088879 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-credential-socket\") pod \"9ee9d700-7952-4277-937c-96af910a747d\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " Jul 22 02:57:09.089085 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.088991 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-workload-socket\") pod \"9ee9d700-7952-4277-937c-96af910a747d\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " Jul 22 02:57:09.089085 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.089014 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ee9d700-7952-4277-937c-96af910a747d-istio-token\") pod \"9ee9d700-7952-4277-937c-96af910a747d\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " Jul 22 02:57:09.089085 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.089029 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-istio-data\") pod \"9ee9d700-7952-4277-937c-96af910a747d\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " Jul 22 02:57:09.089085 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.089059 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsrq6\" (UniqueName: \"kubernetes.io/projected/9ee9d700-7952-4277-937c-96af910a747d-kube-api-access-tsrq6\") pod \"9ee9d700-7952-4277-937c-96af910a747d\" (UID: \"9ee9d700-7952-4277-937c-96af910a747d\") " Jul 22 02:57:09.089457 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.089078 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9ee9d700-7952-4277-937c-96af910a747d" (UID: "9ee9d700-7952-4277-937c-96af910a747d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:57:09.089457 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.089257 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ee9d700-7952-4277-937c-96af910a747d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9ee9d700-7952-4277-937c-96af910a747d" (UID: "9ee9d700-7952-4277-937c-96af910a747d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:57:09.089457 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.089283 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9ee9d700-7952-4277-937c-96af910a747d" (UID: "9ee9d700-7952-4277-937c-96af910a747d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:57:09.089457 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.089395 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-workload-certs\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:57:09.089659 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.089488 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9ee9d700-7952-4277-937c-96af910a747d" (UID: "9ee9d700-7952-4277-937c-96af910a747d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:57:09.090082 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.090058 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-istio-data" (OuterVolumeSpecName: "istio-data") pod "9ee9d700-7952-4277-937c-96af910a747d" (UID: "9ee9d700-7952-4277-937c-96af910a747d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:57:09.091350 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.091313 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ee9d700-7952-4277-937c-96af910a747d-istio-token" (OuterVolumeSpecName: "istio-token") pod "9ee9d700-7952-4277-937c-96af910a747d" (UID: "9ee9d700-7952-4277-937c-96af910a747d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:57:09.091424 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.091354 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9ee9d700-7952-4277-937c-96af910a747d" (UID: "9ee9d700-7952-4277-937c-96af910a747d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:57:09.091851 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.091822 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ee9d700-7952-4277-937c-96af910a747d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9ee9d700-7952-4277-937c-96af910a747d" (UID: "9ee9d700-7952-4277-937c-96af910a747d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:57:09.091917 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.091849 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ee9d700-7952-4277-937c-96af910a747d-kube-api-access-tsrq6" (OuterVolumeSpecName: "kube-api-access-tsrq6") pod "9ee9d700-7952-4277-937c-96af910a747d" (UID: "9ee9d700-7952-4277-937c-96af910a747d"). InnerVolumeSpecName "kube-api-access-tsrq6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:57:09.190235 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.190146 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ee9d700-7952-4277-937c-96af910a747d-istiod-ca-cert\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:57:09.190235 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.190177 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ee9d700-7952-4277-937c-96af910a747d-istio-podinfo\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:57:09.190235 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.190187 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-istio-envoy\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:57:09.190235 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.190195 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-credential-socket\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:57:09.190235 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.190204 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-workload-socket\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:57:09.190235 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.190213 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ee9d700-7952-4277-937c-96af910a747d-istio-token\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:57:09.190235 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.190222 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ee9d700-7952-4277-937c-96af910a747d-istio-data\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:57:09.190235 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.190230 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsrq6\" (UniqueName: \"kubernetes.io/projected/9ee9d700-7952-4277-937c-96af910a747d-kube-api-access-tsrq6\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:57:09.209778 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.209742 2579 generic.go:356] "Generic (PLEG): container finished" podID="9ee9d700-7952-4277-937c-96af910a747d" containerID="9af52cc2caa5e17b6624e94b3ddc47466701f0e34b788f34ffb181f350ec9f19" exitCode=0 Jul 22 02:57:09.209964 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.209795 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" Jul 22 02:57:09.209964 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.209827 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" event={"ID":"9ee9d700-7952-4277-937c-96af910a747d","Type":"ContainerDied","Data":"9af52cc2caa5e17b6624e94b3ddc47466701f0e34b788f34ffb181f350ec9f19"} Jul 22 02:57:09.209964 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.209884 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v" event={"ID":"9ee9d700-7952-4277-937c-96af910a747d","Type":"ContainerDied","Data":"4ba2947f2d95e9ef8526b67212d77083c35351be2d00d692a2ae58773d61fa19"} Jul 22 02:57:09.209964 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.209906 2579 scope.go:117] "RemoveContainer" containerID="9af52cc2caa5e17b6624e94b3ddc47466701f0e34b788f34ffb181f350ec9f19" Jul 22 02:57:09.220727 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.220671 2579 scope.go:117] "RemoveContainer" containerID="bef123d1373f075656ecde22741eced6072b1b8e88ef3b697d598b8bf6cc40bf" Jul 22 02:57:09.229900 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.229864 2579 scope.go:117] "RemoveContainer" containerID="9af52cc2caa5e17b6624e94b3ddc47466701f0e34b788f34ffb181f350ec9f19" Jul 22 02:57:09.230295 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:57:09.230271 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9af52cc2caa5e17b6624e94b3ddc47466701f0e34b788f34ffb181f350ec9f19\": container with ID starting with 9af52cc2caa5e17b6624e94b3ddc47466701f0e34b788f34ffb181f350ec9f19 not found: ID does not exist" containerID="9af52cc2caa5e17b6624e94b3ddc47466701f0e34b788f34ffb181f350ec9f19" Jul 22 02:57:09.230397 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.230309 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9af52cc2caa5e17b6624e94b3ddc47466701f0e34b788f34ffb181f350ec9f19"} err="failed to get container status \"9af52cc2caa5e17b6624e94b3ddc47466701f0e34b788f34ffb181f350ec9f19\": rpc error: code = NotFound desc = could not find container \"9af52cc2caa5e17b6624e94b3ddc47466701f0e34b788f34ffb181f350ec9f19\": container with ID starting with 9af52cc2caa5e17b6624e94b3ddc47466701f0e34b788f34ffb181f350ec9f19 not found: ID does not exist" Jul 22 02:57:09.230397 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.230339 2579 scope.go:117] "RemoveContainer" containerID="bef123d1373f075656ecde22741eced6072b1b8e88ef3b697d598b8bf6cc40bf" Jul 22 02:57:09.230732 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:57:09.230709 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bef123d1373f075656ecde22741eced6072b1b8e88ef3b697d598b8bf6cc40bf\": container with ID starting with bef123d1373f075656ecde22741eced6072b1b8e88ef3b697d598b8bf6cc40bf not found: ID does not exist" containerID="bef123d1373f075656ecde22741eced6072b1b8e88ef3b697d598b8bf6cc40bf" Jul 22 02:57:09.230805 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.230740 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bef123d1373f075656ecde22741eced6072b1b8e88ef3b697d598b8bf6cc40bf"} err="failed to get container status \"bef123d1373f075656ecde22741eced6072b1b8e88ef3b697d598b8bf6cc40bf\": rpc error: code = NotFound desc = could not find container \"bef123d1373f075656ecde22741eced6072b1b8e88ef3b697d598b8bf6cc40bf\": container with ID starting with bef123d1373f075656ecde22741eced6072b1b8e88ef3b697d598b8bf6cc40bf not found: ID does not exist" Jul 22 02:57:09.233208 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.233187 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v"] Jul 22 02:57:09.237045 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.237016 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-113f7429-gw-openshift-default-668ff44475-ss87v"] Jul 22 02:57:09.630093 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:09.630059 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ee9d700-7952-4277-937c-96af910a747d" path="/var/lib/kubelet/pods/9ee9d700-7952-4277-937c-96af910a747d/volumes" Jul 22 02:57:32.904141 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:32.904102 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49fcd44d-s844d"] Jul 22 02:57:32.904527 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:32.904505 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ee9d700-7952-4277-937c-96af910a747d" containerName="istio-proxy" Jul 22 02:57:32.904527 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:32.904516 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ee9d700-7952-4277-937c-96af910a747d" containerName="istio-proxy" Jul 22 02:57:32.904601 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:32.904595 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ee9d700-7952-4277-937c-96af910a747d" containerName="istio-proxy" Jul 22 02:57:32.904636 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:32.904605 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ee9d700-7952-4277-937c-96af910a747d" containerName="istio-proxy" Jul 22 02:57:32.907747 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:32.907730 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49fcd44d-s844d" Jul 22 02:57:32.910493 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:32.910475 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 02:57:32.910614 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:32.910481 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tm89v\"" Jul 22 02:57:32.914649 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:32.914620 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a038f2fe-6bcd-40b4-b0e4-942583c881e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-49fcd44d-s844d\" (UID: \"a038f2fe-6bcd-40b4-b0e4-942583c881e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49fcd44d-s844d" Jul 22 02:57:32.915740 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:32.915716 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49fcd44d-s844d"] Jul 22 02:57:33.015489 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:33.015449 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a038f2fe-6bcd-40b4-b0e4-942583c881e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-49fcd44d-s844d\" (UID: \"a038f2fe-6bcd-40b4-b0e4-942583c881e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49fcd44d-s844d" Jul 22 02:57:33.016110 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:33.016092 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a038f2fe-6bcd-40b4-b0e4-942583c881e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-49fcd44d-s844d\" (UID: \"a038f2fe-6bcd-40b4-b0e4-942583c881e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49fcd44d-s844d" Jul 22 02:57:33.219716 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:33.219631 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49fcd44d-s844d" Jul 22 02:57:33.367056 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:33.367026 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49fcd44d-s844d"] Jul 22 02:57:33.369456 ip-10-0-130-249 kubenswrapper[2579]: W0722 02:57:33.369421 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda038f2fe_6bcd_40b4_b0e4_942583c881e7.slice/crio-bc9cd9421ead7786ecd36e28cdf1a5eeef93912f501894e7433ea6e2f061b5aa WatchSource:0}: Error finding container bc9cd9421ead7786ecd36e28cdf1a5eeef93912f501894e7433ea6e2f061b5aa: Status 404 returned error can't find the container with id bc9cd9421ead7786ecd36e28cdf1a5eeef93912f501894e7433ea6e2f061b5aa Jul 22 02:57:34.350038 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:34.350007 2579 generic.go:356] "Generic (PLEG): container finished" podID="a038f2fe-6bcd-40b4-b0e4-942583c881e7" containerID="b1e849aea0fb0eb8c1ed9263e71514d0d4282b67c39440441f6fe67cdb94e458" exitCode=0 Jul 22 02:57:34.350414 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:34.350074 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49fcd44d-s844d" event={"ID":"a038f2fe-6bcd-40b4-b0e4-942583c881e7","Type":"ContainerDied","Data":"b1e849aea0fb0eb8c1ed9263e71514d0d4282b67c39440441f6fe67cdb94e458"} Jul 22 02:57:34.350414 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:34.350114 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49fcd44d-s844d" event={"ID":"a038f2fe-6bcd-40b4-b0e4-942583c881e7","Type":"ContainerStarted","Data":"bc9cd9421ead7786ecd36e28cdf1a5eeef93912f501894e7433ea6e2f061b5aa"} Jul 22 02:57:35.503618 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:35.503593 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49fcd44d-s844d" Jul 22 02:57:35.533828 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:35.533797 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a038f2fe-6bcd-40b4-b0e4-942583c881e7-maas-api-service-ca\") pod \"a038f2fe-6bcd-40b4-b0e4-942583c881e7\" (UID: \"a038f2fe-6bcd-40b4-b0e4-942583c881e7\") " Jul 22 02:57:35.534156 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:35.534131 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a038f2fe-6bcd-40b4-b0e4-942583c881e7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a038f2fe-6bcd-40b4-b0e4-942583c881e7" (UID: "a038f2fe-6bcd-40b4-b0e4-942583c881e7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:57:35.634679 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:35.634582 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a038f2fe-6bcd-40b4-b0e4-942583c881e7-maas-api-service-ca\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:57:36.362627 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:36.362596 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49fcd44d-s844d" Jul 22 02:57:36.362814 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:36.362599 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49fcd44d-s844d" event={"ID":"a038f2fe-6bcd-40b4-b0e4-942583c881e7","Type":"ContainerDied","Data":"bc9cd9421ead7786ecd36e28cdf1a5eeef93912f501894e7433ea6e2f061b5aa"} Jul 22 02:57:36.362814 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:36.362752 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc9cd9421ead7786ecd36e28cdf1a5eeef93912f501894e7433ea6e2f061b5aa" Jul 22 02:57:43.372531 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:43.372483 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:47.983210 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:47.983095 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49fcd44d-s844d"] Jul 22 02:57:47.985579 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:47.985547 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:47.985717 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:47.985629 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49fcd44d-s844d"] Jul 22 02:57:49.231130 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:49.231075 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:49.468073 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:49.468036 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:49.628062 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:49.628034 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a038f2fe-6bcd-40b4-b0e4-942583c881e7" path="/var/lib/kubelet/pods/a038f2fe-6bcd-40b4-b0e4-942583c881e7/volumes" Jul 22 02:57:49.684698 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:49.684658 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:49.967650 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:49.967555 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8"] Jul 22 02:57:49.968136 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:49.968105 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ee9d700-7952-4277-937c-96af910a747d" containerName="istio-proxy" Jul 22 02:57:49.968136 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:49.968129 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ee9d700-7952-4277-937c-96af910a747d" containerName="istio-proxy" Jul 22 02:57:49.968136 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:49.968145 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a038f2fe-6bcd-40b4-b0e4-942583c881e7" containerName="revoke-keys" Jul 22 02:57:49.968350 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:49.968154 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a038f2fe-6bcd-40b4-b0e4-942583c881e7" containerName="revoke-keys" Jul 22 02:57:49.968350 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:49.968246 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a038f2fe-6bcd-40b4-b0e4-942583c881e7" containerName="revoke-keys" Jul 22 02:57:49.971726 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:49.971706 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:49.974438 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:49.974416 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-e4d739e2-openshift-default-dockercfg-tp8hc\"" Jul 22 02:57:49.982065 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:49.982039 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8"] Jul 22 02:57:49.988540 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:49.988511 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:50.069105 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.069068 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-data\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.069293 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.069116 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwbps\" (UniqueName: \"kubernetes.io/projected/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-kube-api-access-jwbps\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.069293 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.069197 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-podinfo\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.069293 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.069277 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-workload-socket\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.069405 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.069302 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-token\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.069405 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.069345 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-credential-socket\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.069405 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.069370 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-envoy\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.069405 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.069401 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-workload-certs\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.069533 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.069437 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istiod-ca-cert\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.086972 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.086924 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:50.170256 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.170209 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-credential-socket\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.170256 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.170257 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-envoy\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.170512 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.170279 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-workload-certs\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.170512 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.170397 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istiod-ca-cert\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.170512 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.170461 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-data\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.170512 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.170483 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwbps\" (UniqueName: \"kubernetes.io/projected/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-kube-api-access-jwbps\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.170727 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.170546 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-podinfo\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.170908 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.170874 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-workload-socket\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.171036 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.170921 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-token\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.171036 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.170786 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-credential-socket\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.171036 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.170738 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-workload-certs\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.171184 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.171141 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-workload-socket\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.171253 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.171239 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-data\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.171442 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.171421 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istiod-ca-cert\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.172799 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.172774 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-envoy\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.173332 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.173312 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-podinfo\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.180313 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.180285 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-token\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.180459 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.180362 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwbps\" (UniqueName: \"kubernetes.io/projected/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-kube-api-access-jwbps\") pod \"e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.284416 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.284326 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:50.435198 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.435153 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8"] Jul 22 02:57:50.446177 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.446126 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:57:50.446291 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.446194 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:57:50.446291 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:50.446226 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:57:51.449693 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:51.449655 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" event={"ID":"3c471fd3-78bc-433d-b700-c75f7dcdf8c7","Type":"ContainerStarted","Data":"4da59a6362fd18d808e76b639c2fc11bc65e4c502667574d80d90018912bf7fb"} Jul 22 02:57:51.450126 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:51.449715 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" event={"ID":"3c471fd3-78bc-433d-b700-c75f7dcdf8c7","Type":"ContainerStarted","Data":"84a1fcb7ef8c39f0ca08e9d8dd7f96fa0575c22c8b01577dc8c02fd5a1d826af"} Jul 22 02:57:51.473293 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:51.473239 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" podStartSLOduration=2.47322358 podStartE2EDuration="2.47322358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:57:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:57:51.470792301 +0000 UTC m=+1796.319649134" watchObservedRunningTime="2026-07-22 02:57:51.47322358 +0000 UTC m=+1796.322080412" Jul 22 02:57:52.284842 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:52.284805 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:52.289680 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:52.289651 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:52.455414 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:52.455369 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:52.456443 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:52.456427 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:57:52.816403 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:52.816368 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:52.903566 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:52.903498 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:52.979063 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:52.979028 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:53.117593 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:53.117563 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:53.510298 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:53.510218 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:53.611899 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:53.611870 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:57:55.733103 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:55.733077 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 02:57:55.739259 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:55.739237 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 02:57:55.739925 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:55.739909 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 02:57:55.746096 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:57:55.746079 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 02:58:05.535313 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:05.535215 2579 generic.go:356] "Generic (PLEG): container finished" podID="3c471fd3-78bc-433d-b700-c75f7dcdf8c7" containerID="4da59a6362fd18d808e76b639c2fc11bc65e4c502667574d80d90018912bf7fb" exitCode=0 Jul 22 02:58:05.535313 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:05.535287 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" event={"ID":"3c471fd3-78bc-433d-b700-c75f7dcdf8c7","Type":"ContainerDied","Data":"4da59a6362fd18d808e76b639c2fc11bc65e4c502667574d80d90018912bf7fb"} Jul 22 02:58:05.535756 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:05.535676 2579 scope.go:117] "RemoveContainer" containerID="4da59a6362fd18d808e76b639c2fc11bc65e4c502667574d80d90018912bf7fb" Jul 22 02:58:05.536704 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:05.536680 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:58:05.536772 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:05.536745 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:58:05.536811 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:05.536778 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 02:58:06.543418 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:06.543375 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" event={"ID":"3c471fd3-78bc-433d-b700-c75f7dcdf8c7","Type":"ContainerStarted","Data":"511d4d0eefab8037795f92edcd6375831d5a2defc830b4e99b5a2e1c787bb5fc"} Jul 22 02:58:07.285921 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:07.285874 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:58:07.286168 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:07.286100 2579 patch_prober.go:28] interesting pod/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 22 02:58:07.286168 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:07.286147 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" podUID="3c471fd3-78bc-433d-b700-c75f7dcdf8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 22 02:58:07.455920 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:07.455875 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:58:08.284805 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:08.284765 2579 patch_prober.go:28] interesting pod/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 22 02:58:08.285467 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:08.285431 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" podUID="3c471fd3-78bc-433d-b700-c75f7dcdf8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 22 02:58:09.285666 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:09.285629 2579 patch_prober.go:28] interesting pod/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 22 02:58:09.286254 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:09.286223 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" podUID="3c471fd3-78bc-433d-b700-c75f7dcdf8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 22 02:58:10.285785 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:10.285296 2579 patch_prober.go:28] interesting pod/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 22 02:58:10.285785 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:10.285377 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" podUID="3c471fd3-78bc-433d-b700-c75f7dcdf8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 22 02:58:11.288907 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:11.288870 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:58:11.290158 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:11.290139 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:58:14.299760 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:14.299710 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:14.402590 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:14.402551 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:15.452539 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:15.452507 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:15.553484 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:15.553445 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:28.274341 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:28.274266 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:32.873665 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:32.873622 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:33.038375 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:33.038342 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8"] Jul 22 02:58:33.038559 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:33.038538 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" podUID="3c471fd3-78bc-433d-b700-c75f7dcdf8c7" containerName="istio-proxy" containerID="cri-o://511d4d0eefab8037795f92edcd6375831d5a2defc830b4e99b5a2e1c787bb5fc" gracePeriod=30 Jul 22 02:58:33.084074 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:33.084033 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:33.386684 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:33.386638 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:33.860611 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:33.860580 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:33.956543 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:33.956509 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:34.068534 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:34.068502 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:35.746613 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:35.746582 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:38.304185 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.304111 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:58:38.434851 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.434812 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwbps\" (UniqueName: \"kubernetes.io/projected/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-kube-api-access-jwbps\") pod \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " Jul 22 02:58:38.435090 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.434890 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-data\") pod \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " Jul 22 02:58:38.435090 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.434907 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-workload-certs\") pod \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " Jul 22 02:58:38.435090 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.435049 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-workload-socket\") pod \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " Jul 22 02:58:38.435090 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.435086 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-token\") pod \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " Jul 22 02:58:38.435315 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.435157 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-credential-socket\") pod \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " Jul 22 02:58:38.435315 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.435182 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-envoy\") pod \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " Jul 22 02:58:38.435315 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.435185 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3c471fd3-78bc-433d-b700-c75f7dcdf8c7" (UID: "3c471fd3-78bc-433d-b700-c75f7dcdf8c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:38.435315 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.435205 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istiod-ca-cert\") pod \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " Jul 22 02:58:38.435315 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.435245 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-podinfo\") pod \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\" (UID: \"3c471fd3-78bc-433d-b700-c75f7dcdf8c7\") " Jul 22 02:58:38.435611 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.435342 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3c471fd3-78bc-433d-b700-c75f7dcdf8c7" (UID: "3c471fd3-78bc-433d-b700-c75f7dcdf8c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:38.435611 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.435427 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3c471fd3-78bc-433d-b700-c75f7dcdf8c7" (UID: "3c471fd3-78bc-433d-b700-c75f7dcdf8c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:38.435718 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.435643 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-workload-certs\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:58:38.435718 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.435671 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-workload-socket\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:58:38.435718 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.435678 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3c471fd3-78bc-433d-b700-c75f7dcdf8c7" (UID: "3c471fd3-78bc-433d-b700-c75f7dcdf8c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:58:38.435718 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.435686 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-credential-socket\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:58:38.436245 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.436222 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-data" (OuterVolumeSpecName: "istio-data") pod "3c471fd3-78bc-433d-b700-c75f7dcdf8c7" (UID: "3c471fd3-78bc-433d-b700-c75f7dcdf8c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:38.437585 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.437562 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-kube-api-access-jwbps" (OuterVolumeSpecName: "kube-api-access-jwbps") pod "3c471fd3-78bc-433d-b700-c75f7dcdf8c7" (UID: "3c471fd3-78bc-433d-b700-c75f7dcdf8c7"). InnerVolumeSpecName "kube-api-access-jwbps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:58:38.437685 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.437622 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3c471fd3-78bc-433d-b700-c75f7dcdf8c7" (UID: "3c471fd3-78bc-433d-b700-c75f7dcdf8c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:58:38.437767 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.437745 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-token" (OuterVolumeSpecName: "istio-token") pod "3c471fd3-78bc-433d-b700-c75f7dcdf8c7" (UID: "3c471fd3-78bc-433d-b700-c75f7dcdf8c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:58:38.437812 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.437752 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3c471fd3-78bc-433d-b700-c75f7dcdf8c7" (UID: "3c471fd3-78bc-433d-b700-c75f7dcdf8c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:38.536966 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.536909 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwbps\" (UniqueName: \"kubernetes.io/projected/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-kube-api-access-jwbps\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:58:38.536966 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.536965 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-data\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:58:38.536966 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.536975 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-token\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:58:38.537247 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.536986 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-envoy\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:58:38.537247 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.537007 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istiod-ca-cert\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:58:38.537247 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.537015 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c471fd3-78bc-433d-b700-c75f7dcdf8c7-istio-podinfo\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 02:58:38.716335 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.716303 2579 generic.go:356] "Generic (PLEG): container finished" podID="3c471fd3-78bc-433d-b700-c75f7dcdf8c7" containerID="511d4d0eefab8037795f92edcd6375831d5a2defc830b4e99b5a2e1c787bb5fc" exitCode=0 Jul 22 02:58:38.716520 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.716356 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" Jul 22 02:58:38.716520 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.716391 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" event={"ID":"3c471fd3-78bc-433d-b700-c75f7dcdf8c7","Type":"ContainerDied","Data":"511d4d0eefab8037795f92edcd6375831d5a2defc830b4e99b5a2e1c787bb5fc"} Jul 22 02:58:38.716520 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.716451 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8" event={"ID":"3c471fd3-78bc-433d-b700-c75f7dcdf8c7","Type":"ContainerDied","Data":"84a1fcb7ef8c39f0ca08e9d8dd7f96fa0575c22c8b01577dc8c02fd5a1d826af"} Jul 22 02:58:38.716520 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.716480 2579 scope.go:117] "RemoveContainer" containerID="511d4d0eefab8037795f92edcd6375831d5a2defc830b4e99b5a2e1c787bb5fc" Jul 22 02:58:38.726386 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.726366 2579 scope.go:117] "RemoveContainer" containerID="4da59a6362fd18d808e76b639c2fc11bc65e4c502667574d80d90018912bf7fb" Jul 22 02:58:38.734996 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.734937 2579 scope.go:117] "RemoveContainer" containerID="511d4d0eefab8037795f92edcd6375831d5a2defc830b4e99b5a2e1c787bb5fc" Jul 22 02:58:38.735280 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:58:38.735256 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"511d4d0eefab8037795f92edcd6375831d5a2defc830b4e99b5a2e1c787bb5fc\": container with ID starting with 511d4d0eefab8037795f92edcd6375831d5a2defc830b4e99b5a2e1c787bb5fc not found: ID does not exist" containerID="511d4d0eefab8037795f92edcd6375831d5a2defc830b4e99b5a2e1c787bb5fc" Jul 22 02:58:38.735382 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.735284 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"511d4d0eefab8037795f92edcd6375831d5a2defc830b4e99b5a2e1c787bb5fc"} err="failed to get container status \"511d4d0eefab8037795f92edcd6375831d5a2defc830b4e99b5a2e1c787bb5fc\": rpc error: code = NotFound desc = could not find container \"511d4d0eefab8037795f92edcd6375831d5a2defc830b4e99b5a2e1c787bb5fc\": container with ID starting with 511d4d0eefab8037795f92edcd6375831d5a2defc830b4e99b5a2e1c787bb5fc not found: ID does not exist" Jul 22 02:58:38.735382 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.735302 2579 scope.go:117] "RemoveContainer" containerID="4da59a6362fd18d808e76b639c2fc11bc65e4c502667574d80d90018912bf7fb" Jul 22 02:58:38.735507 ip-10-0-130-249 kubenswrapper[2579]: E0722 02:58:38.735492 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4da59a6362fd18d808e76b639c2fc11bc65e4c502667574d80d90018912bf7fb\": container with ID starting with 4da59a6362fd18d808e76b639c2fc11bc65e4c502667574d80d90018912bf7fb not found: ID does not exist" containerID="4da59a6362fd18d808e76b639c2fc11bc65e4c502667574d80d90018912bf7fb" Jul 22 02:58:38.735549 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.735513 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4da59a6362fd18d808e76b639c2fc11bc65e4c502667574d80d90018912bf7fb"} err="failed to get container status \"4da59a6362fd18d808e76b639c2fc11bc65e4c502667574d80d90018912bf7fb\": rpc error: code = NotFound desc = could not find container \"4da59a6362fd18d808e76b639c2fc11bc65e4c502667574d80d90018912bf7fb\": container with ID starting with 4da59a6362fd18d808e76b639c2fc11bc65e4c502667574d80d90018912bf7fb not found: ID does not exist" Jul 22 02:58:38.744462 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.744436 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8"] Jul 22 02:58:38.751494 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:38.751469 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-e4d739e2-openshift-default-8cb5d5d6c-tmgw8"] Jul 22 02:58:39.628059 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:39.628027 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c471fd3-78bc-433d-b700-c75f7dcdf8c7" path="/var/lib/kubelet/pods/3c471fd3-78bc-433d-b700-c75f7dcdf8c7/volumes" Jul 22 02:58:41.961128 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:41.961094 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:42.072981 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:42.072926 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:42.160989 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:42.160922 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:42.261260 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:42.261179 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:42.709663 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:58:42.709630 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:59:08.705435 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:59:08.705398 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:59:15.780903 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:59:15.780862 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:59:15.924340 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:59:15.924305 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:59:16.135781 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:59:16.135748 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:59:32.011147 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:59:32.011109 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:59:37.593858 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:59:37.593818 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:59:37.653834 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:59:37.653799 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:59:37.738192 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:59:37.738153 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:59:37.835262 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:59:37.835226 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:59:48.333422 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:59:48.333383 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:59:49.153209 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:59:49.153175 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:59:55.433185 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:59:55.433151 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:59:55.605892 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:59:55.605847 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:59:55.890424 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:59:55.890385 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:59:56.765938 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:59:56.765894 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 02:59:56.861214 ip-10-0-130-249 kubenswrapper[2579]: I0722 02:59:56.861178 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:03.737302 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:03.737264 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:09.559797 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:09.559764 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:09.659928 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:09.659893 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:10.245682 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:10.245641 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:16.078853 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.078798 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:16.179065 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.179028 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:16.323300 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.323254 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:16.599959 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.599913 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c"] Jul 22 03:00:16.600388 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.600373 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c471fd3-78bc-433d-b700-c75f7dcdf8c7" containerName="istio-proxy" Jul 22 03:00:16.600388 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.600388 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c471fd3-78bc-433d-b700-c75f7dcdf8c7" containerName="istio-proxy" Jul 22 03:00:16.600494 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.600480 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c471fd3-78bc-433d-b700-c75f7dcdf8c7" containerName="istio-proxy" Jul 22 03:00:16.603773 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.603749 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.606537 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.606517 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-08162364-openshift-default-dockercfg-cmdgz\"" Jul 22 03:00:16.618914 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.618884 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c"] Jul 22 03:00:16.661361 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.661313 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-credential-socket\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.661556 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.661416 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istiod-ca-cert\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.661556 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.661455 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qxn6\" (UniqueName: \"kubernetes.io/projected/2822ce8b-0234-4f8e-b3d9-15a6632514f9-kube-api-access-8qxn6\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.661918 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.661886 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-workload-certs\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.662107 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.661965 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-data\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.662107 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.662059 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-podinfo\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.662107 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.662101 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-workload-socket\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.662242 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.662186 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-token\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.662242 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.662234 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-envoy\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.763446 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.763401 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-token\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.763662 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.763469 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-envoy\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.763662 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.763522 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-credential-socket\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.763662 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.763544 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istiod-ca-cert\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.763662 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.763568 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qxn6\" (UniqueName: \"kubernetes.io/projected/2822ce8b-0234-4f8e-b3d9-15a6632514f9-kube-api-access-8qxn6\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.763888 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.763817 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-workload-certs\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.763989 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.763882 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-data\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.763989 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.763935 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-podinfo\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.764112 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.764004 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-workload-socket\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.764599 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.764236 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-credential-socket\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.764599 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.764268 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-data\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.764599 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.764514 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-workload-socket\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.764819 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.764626 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-workload-certs\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.765021 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.764996 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istiod-ca-cert\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.766636 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.766610 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-envoy\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.767219 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.767196 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-podinfo\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.775769 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.775743 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-token\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.778026 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.778004 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qxn6\" (UniqueName: \"kubernetes.io/projected/2822ce8b-0234-4f8e-b3d9-15a6632514f9-kube-api-access-8qxn6\") pod \"e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:16.915187 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:16.915100 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:17.076589 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:17.076560 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c"] Jul 22 03:00:17.078311 ip-10-0-130-249 kubenswrapper[2579]: W0722 03:00:17.078281 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2822ce8b_0234_4f8e_b3d9_15a6632514f9.slice/crio-8f02c3cde06334cd1ee6dc61869f545680bd2e8a8edf7d15cdb6511fd69ce86d WatchSource:0}: Error finding container 8f02c3cde06334cd1ee6dc61869f545680bd2e8a8edf7d15cdb6511fd69ce86d: Status 404 returned error can't find the container with id 8f02c3cde06334cd1ee6dc61869f545680bd2e8a8edf7d15cdb6511fd69ce86d Jul 22 03:00:17.080453 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:17.080419 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:00:17.080821 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:17.080502 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:00:17.080821 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:17.080548 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:00:17.176600 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:17.176514 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:17.264364 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:17.264329 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" event={"ID":"2822ce8b-0234-4f8e-b3d9-15a6632514f9","Type":"ContainerStarted","Data":"03e0ca5002c4f726769c1f034201d42074501ae0817fa209e9af7fc6a5dc3c0d"} Jul 22 03:00:17.264561 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:17.264400 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" event={"ID":"2822ce8b-0234-4f8e-b3d9-15a6632514f9","Type":"ContainerStarted","Data":"8f02c3cde06334cd1ee6dc61869f545680bd2e8a8edf7d15cdb6511fd69ce86d"} Jul 22 03:00:17.283054 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:17.283002 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:17.289869 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:17.289814 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" podStartSLOduration=1.289792165 podStartE2EDuration="1.289792165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:00:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:00:17.287430383 +0000 UTC m=+1942.136287195" watchObservedRunningTime="2026-07-22 03:00:17.289792165 +0000 UTC m=+1942.138648999" Jul 22 03:00:17.916204 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:17.916164 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:18.920336 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:18.920295 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:19.275580 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:19.275481 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:19.276688 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:19.276668 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:19.588453 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:19.588415 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:19.685303 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:19.685253 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:19.760093 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:19.760056 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:20.301895 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:20.301864 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:20.401209 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:20.401166 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:20.502287 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:20.502250 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:24.140569 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:24.140529 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:24.944918 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:24.944879 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:25.108099 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:25.108061 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:25.407186 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:25.407150 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:28.913853 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:28.913764 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:30.606860 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:30.606819 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:31.187455 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:31.187400 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:32.291674 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:32.291636 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:32.349372 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:32.349336 2579 generic.go:356] "Generic (PLEG): container finished" podID="2822ce8b-0234-4f8e-b3d9-15a6632514f9" containerID="03e0ca5002c4f726769c1f034201d42074501ae0817fa209e9af7fc6a5dc3c0d" exitCode=0 Jul 22 03:00:32.349552 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:32.349407 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" event={"ID":"2822ce8b-0234-4f8e-b3d9-15a6632514f9","Type":"ContainerDied","Data":"03e0ca5002c4f726769c1f034201d42074501ae0817fa209e9af7fc6a5dc3c0d"} Jul 22 03:00:32.349920 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:32.349889 2579 scope.go:117] "RemoveContainer" containerID="03e0ca5002c4f726769c1f034201d42074501ae0817fa209e9af7fc6a5dc3c0d" Jul 22 03:00:32.351255 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:32.351222 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:00:32.351349 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:32.351293 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:00:32.351349 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:32.351328 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:00:33.357007 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:33.356969 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" event={"ID":"2822ce8b-0234-4f8e-b3d9-15a6632514f9","Type":"ContainerStarted","Data":"4fa1f37792a138226a45730377b199f8a898faa7a93292615e4b8d0c1d8ea395"} Jul 22 03:00:33.915851 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:33.915814 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:33.916987 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:33.916934 2579 patch_prober.go:28] interesting pod/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 22 03:00:33.917067 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:33.917016 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" podUID="2822ce8b-0234-4f8e-b3d9-15a6632514f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 22 03:00:34.275982 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:34.275853 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:34.916041 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:34.916001 2579 patch_prober.go:28] interesting pod/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 22 03:00:34.916518 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:34.916056 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" podUID="2822ce8b-0234-4f8e-b3d9-15a6632514f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 22 03:00:35.916165 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:35.916123 2579 patch_prober.go:28] interesting pod/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 22 03:00:35.916556 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:35.916183 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" podUID="2822ce8b-0234-4f8e-b3d9-15a6632514f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 22 03:00:36.915560 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:36.915518 2579 patch_prober.go:28] interesting pod/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 22 03:00:36.915753 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:36.915577 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" podUID="2822ce8b-0234-4f8e-b3d9-15a6632514f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 22 03:00:37.919646 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:37.919610 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:37.920777 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:37.920758 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:42.075006 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:42.074971 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:46.620328 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:46.620241 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:47.021415 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:47.021330 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:53.250310 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:53.250274 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:53.313975 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:53.313902 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:53.479830 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:53.479796 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c"] Jul 22 03:00:53.480122 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:53.480058 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" podUID="2822ce8b-0234-4f8e-b3d9-15a6632514f9" containerName="istio-proxy" containerID="cri-o://4fa1f37792a138226a45730377b199f8a898faa7a93292615e4b8d0c1d8ea395" gracePeriod=30 Jul 22 03:00:53.507344 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:53.507262 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:53.742414 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:53.742378 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:53.867972 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:53.867925 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:53.963045 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:53.963005 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:54.079427 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:54.079385 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:00:58.754617 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.754551 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:58.832402 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.832361 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-envoy\") pod \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " Jul 22 03:00:58.832606 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.832409 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-podinfo\") pod \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " Jul 22 03:00:58.832606 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.832464 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-credential-socket\") pod \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " Jul 22 03:00:58.832606 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.832502 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-data\") pod \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " Jul 22 03:00:58.832606 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.832525 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-workload-certs\") pod \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " Jul 22 03:00:58.832822 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.832653 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-workload-socket\") pod \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " Jul 22 03:00:58.832822 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.832699 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-token\") pod \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " Jul 22 03:00:58.832822 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.832744 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qxn6\" (UniqueName: \"kubernetes.io/projected/2822ce8b-0234-4f8e-b3d9-15a6632514f9-kube-api-access-8qxn6\") pod \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " Jul 22 03:00:58.833028 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.832822 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2822ce8b-0234-4f8e-b3d9-15a6632514f9" (UID: "2822ce8b-0234-4f8e-b3d9-15a6632514f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:00:58.833028 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.832840 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istiod-ca-cert\") pod \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\" (UID: \"2822ce8b-0234-4f8e-b3d9-15a6632514f9\") " Jul 22 03:00:58.833028 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.832872 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2822ce8b-0234-4f8e-b3d9-15a6632514f9" (UID: "2822ce8b-0234-4f8e-b3d9-15a6632514f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:00:58.833028 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.832901 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2822ce8b-0234-4f8e-b3d9-15a6632514f9" (UID: "2822ce8b-0234-4f8e-b3d9-15a6632514f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:00:58.833251 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.833232 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-credential-socket\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:00:58.833312 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.833252 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-workload-certs\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:00:58.833312 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.833266 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-workload-socket\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:00:58.833312 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.833264 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2822ce8b-0234-4f8e-b3d9-15a6632514f9" (UID: "2822ce8b-0234-4f8e-b3d9-15a6632514f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:00:58.833812 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.833781 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-data" (OuterVolumeSpecName: "istio-data") pod "2822ce8b-0234-4f8e-b3d9-15a6632514f9" (UID: "2822ce8b-0234-4f8e-b3d9-15a6632514f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:00:58.834960 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.834918 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2822ce8b-0234-4f8e-b3d9-15a6632514f9" (UID: "2822ce8b-0234-4f8e-b3d9-15a6632514f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:00:58.834960 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.834923 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-token" (OuterVolumeSpecName: "istio-token") pod "2822ce8b-0234-4f8e-b3d9-15a6632514f9" (UID: "2822ce8b-0234-4f8e-b3d9-15a6632514f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:00:58.835101 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.834918 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2822ce8b-0234-4f8e-b3d9-15a6632514f9" (UID: "2822ce8b-0234-4f8e-b3d9-15a6632514f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:00:58.835214 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.835195 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2822ce8b-0234-4f8e-b3d9-15a6632514f9-kube-api-access-8qxn6" (OuterVolumeSpecName: "kube-api-access-8qxn6") pod "2822ce8b-0234-4f8e-b3d9-15a6632514f9" (UID: "2822ce8b-0234-4f8e-b3d9-15a6632514f9"). InnerVolumeSpecName "kube-api-access-8qxn6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:00:58.934314 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.934265 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qxn6\" (UniqueName: \"kubernetes.io/projected/2822ce8b-0234-4f8e-b3d9-15a6632514f9-kube-api-access-8qxn6\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:00:58.934314 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.934310 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istiod-ca-cert\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:00:58.934314 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.934320 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-envoy\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:00:58.934314 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.934329 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-podinfo\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:00:58.934314 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.934336 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-data\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:00:58.934628 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:58.934345 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2822ce8b-0234-4f8e-b3d9-15a6632514f9-istio-token\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:00:59.513649 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:59.513613 2579 generic.go:356] "Generic (PLEG): container finished" podID="2822ce8b-0234-4f8e-b3d9-15a6632514f9" containerID="4fa1f37792a138226a45730377b199f8a898faa7a93292615e4b8d0c1d8ea395" exitCode=0 Jul 22 03:00:59.513649 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:59.513652 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" event={"ID":"2822ce8b-0234-4f8e-b3d9-15a6632514f9","Type":"ContainerDied","Data":"4fa1f37792a138226a45730377b199f8a898faa7a93292615e4b8d0c1d8ea395"} Jul 22 03:00:59.514017 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:59.513674 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" Jul 22 03:00:59.514017 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:59.513696 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c" event={"ID":"2822ce8b-0234-4f8e-b3d9-15a6632514f9","Type":"ContainerDied","Data":"8f02c3cde06334cd1ee6dc61869f545680bd2e8a8edf7d15cdb6511fd69ce86d"} Jul 22 03:00:59.514017 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:59.513719 2579 scope.go:117] "RemoveContainer" containerID="4fa1f37792a138226a45730377b199f8a898faa7a93292615e4b8d0c1d8ea395" Jul 22 03:00:59.523745 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:59.523726 2579 scope.go:117] "RemoveContainer" containerID="03e0ca5002c4f726769c1f034201d42074501ae0817fa209e9af7fc6a5dc3c0d" Jul 22 03:00:59.532581 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:59.532560 2579 scope.go:117] "RemoveContainer" containerID="4fa1f37792a138226a45730377b199f8a898faa7a93292615e4b8d0c1d8ea395" Jul 22 03:00:59.532890 ip-10-0-130-249 kubenswrapper[2579]: E0722 03:00:59.532867 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fa1f37792a138226a45730377b199f8a898faa7a93292615e4b8d0c1d8ea395\": container with ID starting with 4fa1f37792a138226a45730377b199f8a898faa7a93292615e4b8d0c1d8ea395 not found: ID does not exist" containerID="4fa1f37792a138226a45730377b199f8a898faa7a93292615e4b8d0c1d8ea395" Jul 22 03:00:59.532960 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:59.532901 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa1f37792a138226a45730377b199f8a898faa7a93292615e4b8d0c1d8ea395"} err="failed to get container status \"4fa1f37792a138226a45730377b199f8a898faa7a93292615e4b8d0c1d8ea395\": rpc error: code = NotFound desc = could not find container \"4fa1f37792a138226a45730377b199f8a898faa7a93292615e4b8d0c1d8ea395\": container with ID starting with 4fa1f37792a138226a45730377b199f8a898faa7a93292615e4b8d0c1d8ea395 not found: ID does not exist" Jul 22 03:00:59.532960 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:59.532920 2579 scope.go:117] "RemoveContainer" containerID="03e0ca5002c4f726769c1f034201d42074501ae0817fa209e9af7fc6a5dc3c0d" Jul 22 03:00:59.533200 ip-10-0-130-249 kubenswrapper[2579]: E0722 03:00:59.533180 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03e0ca5002c4f726769c1f034201d42074501ae0817fa209e9af7fc6a5dc3c0d\": container with ID starting with 03e0ca5002c4f726769c1f034201d42074501ae0817fa209e9af7fc6a5dc3c0d not found: ID does not exist" containerID="03e0ca5002c4f726769c1f034201d42074501ae0817fa209e9af7fc6a5dc3c0d" Jul 22 03:00:59.533263 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:59.533211 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03e0ca5002c4f726769c1f034201d42074501ae0817fa209e9af7fc6a5dc3c0d"} err="failed to get container status \"03e0ca5002c4f726769c1f034201d42074501ae0817fa209e9af7fc6a5dc3c0d\": rpc error: code = NotFound desc = could not find container \"03e0ca5002c4f726769c1f034201d42074501ae0817fa209e9af7fc6a5dc3c0d\": container with ID starting with 03e0ca5002c4f726769c1f034201d42074501ae0817fa209e9af7fc6a5dc3c0d not found: ID does not exist" Jul 22 03:00:59.536811 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:59.536787 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c"] Jul 22 03:00:59.541048 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:59.541025 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-08162364-openshift-default-5f4456d5cf-jvc5c"] Jul 22 03:00:59.628642 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:59.628608 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2822ce8b-0234-4f8e-b3d9-15a6632514f9" path="/var/lib/kubelet/pods/2822ce8b-0234-4f8e-b3d9-15a6632514f9/volumes" Jul 22 03:00:59.958068 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:00:59.958035 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:01:00.062141 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:00.062106 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:01:00.158493 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:00.158456 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:01:00.313183 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:00.313100 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:01:06.174919 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:06.174877 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:01:06.279323 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:06.279285 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:01:13.062011 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.061967 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:01:13.338393 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.338362 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg"] Jul 22 03:01:13.338796 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.338783 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c471fd3-78bc-433d-b700-c75f7dcdf8c7" containerName="istio-proxy" Jul 22 03:01:13.338841 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.338797 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c471fd3-78bc-433d-b700-c75f7dcdf8c7" containerName="istio-proxy" Jul 22 03:01:13.338841 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.338806 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2822ce8b-0234-4f8e-b3d9-15a6632514f9" containerName="istio-proxy" Jul 22 03:01:13.338841 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.338811 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2822ce8b-0234-4f8e-b3d9-15a6632514f9" containerName="istio-proxy" Jul 22 03:01:13.338841 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.338822 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2822ce8b-0234-4f8e-b3d9-15a6632514f9" containerName="istio-proxy" Jul 22 03:01:13.338841 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.338829 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2822ce8b-0234-4f8e-b3d9-15a6632514f9" containerName="istio-proxy" Jul 22 03:01:13.339025 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.338891 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c471fd3-78bc-433d-b700-c75f7dcdf8c7" containerName="istio-proxy" Jul 22 03:01:13.339025 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.338902 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2822ce8b-0234-4f8e-b3d9-15a6632514f9" containerName="istio-proxy" Jul 22 03:01:13.343691 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.343665 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.347193 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.347167 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ee84bc3c-openshift-default-dockercfg-prhpx\"" Jul 22 03:01:13.358343 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.358312 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg"] Jul 22 03:01:13.368409 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.368373 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:01:13.472321 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.472278 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-credential-socket\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.472321 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.472316 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-token\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.472592 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.472342 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-workload-certs\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.472592 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.472406 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cr6p\" (UniqueName: \"kubernetes.io/projected/49d57d45-9c9f-4157-adf3-1ac72808ae50-kube-api-access-9cr6p\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.472592 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.472467 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-data\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.472592 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.472520 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-podinfo\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.472592 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.472537 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49d57d45-9c9f-4157-adf3-1ac72808ae50-istiod-ca-cert\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.472592 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.472555 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-workload-socket\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.472890 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.472647 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-envoy\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.573875 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.573829 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cr6p\" (UniqueName: \"kubernetes.io/projected/49d57d45-9c9f-4157-adf3-1ac72808ae50-kube-api-access-9cr6p\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.574071 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.573897 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-data\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.574071 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.573938 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-podinfo\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.574071 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.574001 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49d57d45-9c9f-4157-adf3-1ac72808ae50-istiod-ca-cert\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.574071 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.574026 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-workload-socket\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.574291 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.574076 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-envoy\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.574291 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.574131 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-credential-socket\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.574291 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.574156 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-token\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.574291 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.574185 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-workload-certs\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.574498 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.574385 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-data\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.574498 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.574435 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-credential-socket\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.574498 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.574487 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-workload-certs\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.574809 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.574785 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49d57d45-9c9f-4157-adf3-1ac72808ae50-istiod-ca-cert\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.574960 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.574795 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-workload-socket\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.576547 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.576527 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-envoy\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.576918 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.576894 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-podinfo\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.585521 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.585494 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-token\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.585687 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.585662 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cr6p\" (UniqueName: \"kubernetes.io/projected/49d57d45-9c9f-4157-adf3-1ac72808ae50-kube-api-access-9cr6p\") pod \"e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.654746 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.654647 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:13.675767 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.675725 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:01:13.808704 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.808671 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg"] Jul 22 03:01:13.810737 ip-10-0-130-249 kubenswrapper[2579]: W0722 03:01:13.810712 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d57d45_9c9f_4157_adf3_1ac72808ae50.slice/crio-7fedf6c0b97d04860b6d62d0823063590c6997fd754307539d290b8b47e18df9 WatchSource:0}: Error finding container 7fedf6c0b97d04860b6d62d0823063590c6997fd754307539d290b8b47e18df9: Status 404 returned error can't find the container with id 7fedf6c0b97d04860b6d62d0823063590c6997fd754307539d290b8b47e18df9 Jul 22 03:01:13.813587 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.813541 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:01:13.813684 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.813645 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:01:13.813721 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:13.813691 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:01:14.602159 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:14.602123 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" event={"ID":"49d57d45-9c9f-4157-adf3-1ac72808ae50","Type":"ContainerStarted","Data":"4fa9f08c0eb12eb02a71adafdd9b6a343c1a37550e3ddfe363edf691f25deb1f"} Jul 22 03:01:14.602600 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:14.602176 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" event={"ID":"49d57d45-9c9f-4157-adf3-1ac72808ae50","Type":"ContainerStarted","Data":"7fedf6c0b97d04860b6d62d0823063590c6997fd754307539d290b8b47e18df9"} Jul 22 03:01:14.630591 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:14.630541 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" podStartSLOduration=1.6305232570000001 podStartE2EDuration="1.630523257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:01:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:01:14.627366073 +0000 UTC m=+1999.476222906" watchObservedRunningTime="2026-07-22 03:01:14.630523257 +0000 UTC m=+1999.479380089" Jul 22 03:01:14.655663 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:14.655627 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:14.661426 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:14.661397 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:15.608350 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:15.608318 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:15.609438 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:15.609419 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:16.689925 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:16.689891 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:01:16.787537 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:16.787490 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:01:16.916339 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:16.916294 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:01:17.185108 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:17.185072 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:01:17.464220 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:17.464125 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:01:17.552861 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:17.552807 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:01:21.166912 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:21.166862 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:01:30.610348 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:30.610304 2579 patch_prober.go:28] interesting pod/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 22 03:01:30.610852 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:30.610389 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" podUID="49d57d45-9c9f-4157-adf3-1ac72808ae50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 22 03:01:30.698052 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:30.698016 2579 generic.go:356] "Generic (PLEG): container finished" podID="49d57d45-9c9f-4157-adf3-1ac72808ae50" containerID="4fa9f08c0eb12eb02a71adafdd9b6a343c1a37550e3ddfe363edf691f25deb1f" exitCode=0 Jul 22 03:01:30.698223 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:30.698091 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" event={"ID":"49d57d45-9c9f-4157-adf3-1ac72808ae50","Type":"ContainerDied","Data":"4fa9f08c0eb12eb02a71adafdd9b6a343c1a37550e3ddfe363edf691f25deb1f"} Jul 22 03:01:30.698527 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:30.698512 2579 scope.go:117] "RemoveContainer" containerID="4fa9f08c0eb12eb02a71adafdd9b6a343c1a37550e3ddfe363edf691f25deb1f" Jul 22 03:01:30.699659 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:30.699632 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:01:30.699725 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:30.699701 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:01:30.699766 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:30.699746 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:01:31.705441 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:31.705395 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" event={"ID":"49d57d45-9c9f-4157-adf3-1ac72808ae50","Type":"ContainerStarted","Data":"e6edaaf9a495fb7661b0a23788c431a109667c39f002d9386f712cf41bc2635f"} Jul 22 03:01:32.167590 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:32.167548 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:01:32.655052 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:32.655006 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:32.655267 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:32.655244 2579 patch_prober.go:28] interesting pod/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 22 03:01:32.655322 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:32.655295 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" podUID="49d57d45-9c9f-4157-adf3-1ac72808ae50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 22 03:01:33.655626 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:33.655589 2579 patch_prober.go:28] interesting pod/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 22 03:01:33.656080 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:33.655645 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" podUID="49d57d45-9c9f-4157-adf3-1ac72808ae50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 22 03:01:34.656094 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:34.656052 2579 patch_prober.go:28] interesting pod/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 22 03:01:34.656503 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:34.656134 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" podUID="49d57d45-9c9f-4157-adf3-1ac72808ae50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 22 03:01:35.659289 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:35.659256 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:35.659728 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:35.659704 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:01:35.660529 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:01:35.660509 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:02:04.916458 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:02:04.916417 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:02:07.531713 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:02:07.531675 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:02:09.464074 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:02:09.464013 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:02:11.739871 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:02:11.739834 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:02:16.775281 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:02:16.775188 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:02:24.003207 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:02:24.003151 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:02:39.960108 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:02:39.960077 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:02:40.020293 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:02:40.020247 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:02:52.882893 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:02:52.882851 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:02:54.187175 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:02:54.187132 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:02:55.778719 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:02:55.778689 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 03:02:55.783710 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:02:55.783685 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 03:02:55.786688 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:02:55.786668 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 03:02:55.792050 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:02:55.792028 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 03:02:55.851831 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:02:55.851793 2579 scope.go:117] "RemoveContainer" containerID="d632b3b3ad3b843052abbf20935efa4924efd07a8111f044bb9679420a23a5c9" Jul 22 03:02:59.746168 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:02:59.746128 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:02:59.807283 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:02:59.807248 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:00.024359 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:00.024273 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg"] Jul 22 03:03:00.024521 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:00.024482 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" podUID="49d57d45-9c9f-4157-adf3-1ac72808ae50" containerName="istio-proxy" containerID="cri-o://e6edaaf9a495fb7661b0a23788c431a109667c39f002d9386f712cf41bc2635f" gracePeriod=30 Jul 22 03:03:00.050932 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:00.050897 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:00.299814 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:00.299715 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:01.461314 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:01.461266 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:01.555715 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:01.555675 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:01.631299 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:01.631260 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:02.454130 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:02.454090 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:02.554117 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:02.554079 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:02.651231 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:02.651193 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:04.110423 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:04.110384 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:05.235080 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.235046 2579 generic.go:356] "Generic (PLEG): container finished" podID="49d57d45-9c9f-4157-adf3-1ac72808ae50" containerID="e6edaaf9a495fb7661b0a23788c431a109667c39f002d9386f712cf41bc2635f" exitCode=0 Jul 22 03:03:05.235480 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.235123 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" event={"ID":"49d57d45-9c9f-4157-adf3-1ac72808ae50","Type":"ContainerDied","Data":"e6edaaf9a495fb7661b0a23788c431a109667c39f002d9386f712cf41bc2635f"} Jul 22 03:03:05.235480 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.235184 2579 scope.go:117] "RemoveContainer" containerID="4fa9f08c0eb12eb02a71adafdd9b6a343c1a37550e3ddfe363edf691f25deb1f" Jul 22 03:03:05.299889 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.299814 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:03:05.381325 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.381286 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9cr6p\" (UniqueName: \"kubernetes.io/projected/49d57d45-9c9f-4157-adf3-1ac72808ae50-kube-api-access-9cr6p\") pod \"49d57d45-9c9f-4157-adf3-1ac72808ae50\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " Jul 22 03:03:05.381506 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.381340 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49d57d45-9c9f-4157-adf3-1ac72808ae50-istiod-ca-cert\") pod \"49d57d45-9c9f-4157-adf3-1ac72808ae50\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " Jul 22 03:03:05.381506 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.381367 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-data\") pod \"49d57d45-9c9f-4157-adf3-1ac72808ae50\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " Jul 22 03:03:05.381506 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.381404 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-workload-certs\") pod \"49d57d45-9c9f-4157-adf3-1ac72808ae50\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " Jul 22 03:03:05.381506 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.381427 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-envoy\") pod \"49d57d45-9c9f-4157-adf3-1ac72808ae50\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " Jul 22 03:03:05.381506 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.381460 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-podinfo\") pod \"49d57d45-9c9f-4157-adf3-1ac72808ae50\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " Jul 22 03:03:05.381776 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.381521 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-token\") pod \"49d57d45-9c9f-4157-adf3-1ac72808ae50\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " Jul 22 03:03:05.381776 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.381567 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-credential-socket\") pod \"49d57d45-9c9f-4157-adf3-1ac72808ae50\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " Jul 22 03:03:05.381776 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.381599 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-workload-socket\") pod \"49d57d45-9c9f-4157-adf3-1ac72808ae50\" (UID: \"49d57d45-9c9f-4157-adf3-1ac72808ae50\") " Jul 22 03:03:05.381776 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.381747 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "49d57d45-9c9f-4157-adf3-1ac72808ae50" (UID: "49d57d45-9c9f-4157-adf3-1ac72808ae50"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:03:05.382023 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.381856 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49d57d45-9c9f-4157-adf3-1ac72808ae50-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "49d57d45-9c9f-4157-adf3-1ac72808ae50" (UID: "49d57d45-9c9f-4157-adf3-1ac72808ae50"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:03:05.382023 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.381935 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "49d57d45-9c9f-4157-adf3-1ac72808ae50" (UID: "49d57d45-9c9f-4157-adf3-1ac72808ae50"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:03:05.382128 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.382071 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49d57d45-9c9f-4157-adf3-1ac72808ae50-istiod-ca-cert\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:03:05.382128 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.382093 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-workload-certs\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:03:05.382202 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.382140 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "49d57d45-9c9f-4157-adf3-1ac72808ae50" (UID: "49d57d45-9c9f-4157-adf3-1ac72808ae50"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:03:05.382403 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.382383 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-data" (OuterVolumeSpecName: "istio-data") pod "49d57d45-9c9f-4157-adf3-1ac72808ae50" (UID: "49d57d45-9c9f-4157-adf3-1ac72808ae50"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:03:05.384113 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.384082 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-token" (OuterVolumeSpecName: "istio-token") pod "49d57d45-9c9f-4157-adf3-1ac72808ae50" (UID: "49d57d45-9c9f-4157-adf3-1ac72808ae50"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:03:05.384225 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.384126 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "49d57d45-9c9f-4157-adf3-1ac72808ae50" (UID: "49d57d45-9c9f-4157-adf3-1ac72808ae50"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:03:05.384305 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.384287 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "49d57d45-9c9f-4157-adf3-1ac72808ae50" (UID: "49d57d45-9c9f-4157-adf3-1ac72808ae50"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:03:05.384305 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.384286 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49d57d45-9c9f-4157-adf3-1ac72808ae50-kube-api-access-9cr6p" (OuterVolumeSpecName: "kube-api-access-9cr6p") pod "49d57d45-9c9f-4157-adf3-1ac72808ae50" (UID: "49d57d45-9c9f-4157-adf3-1ac72808ae50"). InnerVolumeSpecName "kube-api-access-9cr6p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:03:05.483258 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.483222 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cr6p\" (UniqueName: \"kubernetes.io/projected/49d57d45-9c9f-4157-adf3-1ac72808ae50-kube-api-access-9cr6p\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:03:05.483258 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.483257 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-data\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:03:05.483258 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.483267 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-envoy\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:03:05.483258 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.483275 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-podinfo\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:03:05.483555 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.483283 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49d57d45-9c9f-4157-adf3-1ac72808ae50-istio-token\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:03:05.483555 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.483291 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-credential-socket\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:03:05.483555 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:05.483300 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d57d45-9c9f-4157-adf3-1ac72808ae50-workload-socket\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:03:06.242277 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:06.242246 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" Jul 22 03:03:06.242277 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:06.242262 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg" event={"ID":"49d57d45-9c9f-4157-adf3-1ac72808ae50","Type":"ContainerDied","Data":"7fedf6c0b97d04860b6d62d0823063590c6997fd754307539d290b8b47e18df9"} Jul 22 03:03:06.242805 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:06.242348 2579 scope.go:117] "RemoveContainer" containerID="e6edaaf9a495fb7661b0a23788c431a109667c39f002d9386f712cf41bc2635f" Jul 22 03:03:06.263201 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:06.263160 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg"] Jul 22 03:03:06.270093 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:06.270059 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ee84bc3c-openshift-default-798fbc54d7-cnkhg"] Jul 22 03:03:07.629678 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:07.629642 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49d57d45-9c9f-4157-adf3-1ac72808ae50" path="/var/lib/kubelet/pods/49d57d45-9c9f-4157-adf3-1ac72808ae50/volumes" Jul 22 03:03:09.998054 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:09.998006 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:10.062891 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:10.062851 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:10.680063 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:10.680022 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:15.109341 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:15.109288 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:15.209504 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:15.209473 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:16.632859 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:16.632816 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:16.697721 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:16.697673 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:16.793698 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:16.793668 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:16.893064 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:16.892972 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:27.425316 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:27.425270 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:27.779138 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:27.779033 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:48.747685 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:48.747591 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:55.377364 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:55.377316 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:55.486181 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:55.486144 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:55.746897 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:55.746798 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:55.874989 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:55.874962 2579 scope.go:117] "RemoveContainer" containerID="b1e849aea0fb0eb8c1ed9263e71514d0d4282b67c39440441f6fe67cdb94e458" Jul 22 03:03:56.092880 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.092841 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s"] Jul 22 03:03:56.093304 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.093284 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49d57d45-9c9f-4157-adf3-1ac72808ae50" containerName="istio-proxy" Jul 22 03:03:56.093304 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.093301 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="49d57d45-9c9f-4157-adf3-1ac72808ae50" containerName="istio-proxy" Jul 22 03:03:56.093429 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.093400 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2822ce8b-0234-4f8e-b3d9-15a6632514f9" containerName="istio-proxy" Jul 22 03:03:56.093429 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.093417 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="49d57d45-9c9f-4157-adf3-1ac72808ae50" containerName="istio-proxy" Jul 22 03:03:56.093429 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.093423 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="49d57d45-9c9f-4157-adf3-1ac72808ae50" containerName="istio-proxy" Jul 22 03:03:56.098063 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.098039 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.105808 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.105784 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-317968-openshift-default-dockercfg-2xhwm\"" Jul 22 03:03:56.120207 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.120175 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s"] Jul 22 03:03:56.139532 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.139502 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-envoy\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.139729 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.139547 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istiod-ca-cert\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.139729 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.139574 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-workload-certs\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.139729 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.139627 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-workload-socket\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.139729 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.139682 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-token\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.139976 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.139762 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-credential-socket\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.139976 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.139823 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8dkj\" (UniqueName: \"kubernetes.io/projected/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-kube-api-access-q8dkj\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.139976 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.139887 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-data\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.139976 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.139927 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-podinfo\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.155227 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.155121 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:56.240476 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.240441 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istiod-ca-cert\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.240713 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.240499 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-workload-certs\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.240713 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.240525 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:56.240713 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.240532 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-workload-socket\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.240713 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.240631 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-token\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.240713 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.240675 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-credential-socket\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.241020 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.240725 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8dkj\" (UniqueName: \"kubernetes.io/projected/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-kube-api-access-q8dkj\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.241020 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.240753 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-data\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.241020 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.240796 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-podinfo\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.241020 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.240842 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-envoy\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.241020 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.240919 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-workload-socket\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.241284 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.241264 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istiod-ca-cert\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.241426 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.241403 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-workload-certs\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.241615 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.241468 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-data\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.241615 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.241527 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-credential-socket\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.243294 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.243272 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-envoy\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.243710 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.243691 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-podinfo\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.255433 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.255397 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8dkj\" (UniqueName: \"kubernetes.io/projected/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-kube-api-access-q8dkj\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.255575 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.255399 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-token\") pod \"e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.411087 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.411002 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:56.561593 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.561565 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s"] Jul 22 03:03:56.568543 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.568502 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:03:56.568656 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.568592 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:03:56.568656 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:56.568631 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:03:57.522068 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:57.522025 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" event={"ID":"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268","Type":"ContainerStarted","Data":"e84ea30d9ae7b89f9e788db578e35e494a7a34c86dfdca02168acf71a7b3897d"} Jul 22 03:03:57.522559 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:57.522083 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" event={"ID":"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268","Type":"ContainerStarted","Data":"00114b3672ad15d5e79ece3cb3f8aca99796e5f03ffb6171043f89d3eb17ffcc"} Jul 22 03:03:58.411351 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:58.411306 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:58.416441 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:58.416412 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:58.440850 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:58.440790 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" podStartSLOduration=2.440772558 podStartE2EDuration="2.440772558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:03:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:03:57.54828364 +0000 UTC m=+2162.397140472" watchObservedRunningTime="2026-07-22 03:03:58.440772558 +0000 UTC m=+2163.289629475" Jul 22 03:03:58.528024 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:58.527990 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:58.529028 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:58.529009 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:03:59.574985 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:59.574930 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:59.694922 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:59.694879 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:03:59.795901 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:03:59.795859 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:00.015078 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:00.015028 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:00.524963 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:00.524904 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:00.624694 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:00.624659 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:03.997099 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:03.997060 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:04.342076 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:04.342038 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:04.694759 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:04.694651 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:04.854960 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:04.854905 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:07.919936 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:07.919901 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:08.100715 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:08.100586 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:08.290556 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:08.289892 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:08.497227 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:08.497179 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:09.008153 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:09.008100 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:09.172427 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:09.172394 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:12.450148 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:12.450114 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:12.616199 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:12.616165 2579 generic.go:356] "Generic (PLEG): container finished" podID="7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" containerID="e84ea30d9ae7b89f9e788db578e35e494a7a34c86dfdca02168acf71a7b3897d" exitCode=0 Jul 22 03:04:12.616375 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:12.616234 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" event={"ID":"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268","Type":"ContainerDied","Data":"e84ea30d9ae7b89f9e788db578e35e494a7a34c86dfdca02168acf71a7b3897d"} Jul 22 03:04:12.616650 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:12.616636 2579 scope.go:117] "RemoveContainer" containerID="e84ea30d9ae7b89f9e788db578e35e494a7a34c86dfdca02168acf71a7b3897d" Jul 22 03:04:12.617724 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:12.617695 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:04:12.617862 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:12.617780 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:04:12.617862 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:12.617831 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:04:13.528309 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:13.528269 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:04:13.629170 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:13.629123 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" event={"ID":"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268","Type":"ContainerStarted","Data":"735ca563ef441d61b7e76899b187cfe0812ee93dc9106d450ea5fbbfdbb4d3b9"} Jul 22 03:04:14.411537 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:14.411494 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:04:14.412767 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:14.412741 2579 patch_prober.go:28] interesting pod/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 22 03:04:14.412871 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:14.412806 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" podUID="7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 22 03:04:15.412387 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:15.412353 2579 patch_prober.go:28] interesting pod/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 22 03:04:15.412764 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:15.412409 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" podUID="7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 22 03:04:16.411758 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:16.411721 2579 patch_prober.go:28] interesting pod/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 22 03:04:16.411959 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:16.411807 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" podUID="7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 22 03:04:17.411644 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:17.411600 2579 patch_prober.go:28] interesting pod/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 22 03:04:17.412107 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:17.411665 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" podUID="7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 22 03:04:18.415938 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:18.415903 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:04:18.416503 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:18.416477 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:04:18.417156 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:18.417136 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:04:21.765667 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:21.765629 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:21.925598 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:21.925565 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:23.553050 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:23.553000 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:30.209268 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:30.209230 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:30.315009 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:30.314977 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:04:56.551902 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:04:56.551860 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:05:37.492419 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:05:37.492321 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:05:38.509174 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:05:38.509113 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:05:39.275375 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:05:39.275332 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:05:39.371923 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:05:39.371885 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:05:49.342418 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:05:49.342380 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:05:53.703176 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:05:53.703132 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:05:54.163793 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:05:54.163757 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:01.072533 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:01.072468 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:01.192882 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:01.192849 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:01.406424 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:01.406388 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s"] Jul 22 03:06:01.406630 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:01.406594 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" podUID="7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" containerName="istio-proxy" containerID="cri-o://735ca563ef441d61b7e76899b187cfe0812ee93dc9106d450ea5fbbfdbb4d3b9" gracePeriod=30 Jul 22 03:06:01.464563 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:01.464522 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:01.635636 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:01.635605 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:02.457302 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:02.457250 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:02.702834 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:02.702795 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:03.418596 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:03.418560 2579 patch_prober.go:28] interesting pod/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 03:06:03.418781 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:03.418634 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" podUID="7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 03:06:06.674824 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.674759 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:06:06.817445 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.817409 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-workload-socket\") pod \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " Jul 22 03:06:06.817445 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.817448 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-token\") pod \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " Jul 22 03:06:06.817709 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.817475 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istiod-ca-cert\") pod \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " Jul 22 03:06:06.817709 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.817523 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-data\") pod \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " Jul 22 03:06:06.817709 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.817551 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-podinfo\") pod \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " Jul 22 03:06:06.817709 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.817570 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-workload-certs\") pod \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " Jul 22 03:06:06.817709 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.817596 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-envoy\") pod \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " Jul 22 03:06:06.817709 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.817638 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-credential-socket\") pod \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " Jul 22 03:06:06.817709 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.817662 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8dkj\" (UniqueName: \"kubernetes.io/projected/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-kube-api-access-q8dkj\") pod \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\" (UID: \"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268\") " Jul 22 03:06:06.817709 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.817705 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" (UID: "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:06:06.818151 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.818033 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-workload-socket\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:06:06.818151 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.818055 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" (UID: "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:06:06.818151 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.818093 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" (UID: "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:06:06.818477 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.818207 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" (UID: "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:06:06.818550 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.818529 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-data" (OuterVolumeSpecName: "istio-data") pod "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" (UID: "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:06:06.820032 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.819998 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-token" (OuterVolumeSpecName: "istio-token") pod "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" (UID: "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:06:06.820170 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.820039 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-kube-api-access-q8dkj" (OuterVolumeSpecName: "kube-api-access-q8dkj") pod "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" (UID: "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268"). InnerVolumeSpecName "kube-api-access-q8dkj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:06:06.820366 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.820346 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" (UID: "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:06:06.820437 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.820373 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" (UID: "7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:06:06.919331 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.919279 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-data\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:06:06.919331 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.919309 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-podinfo\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:06:06.919331 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.919319 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-workload-certs\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:06:06.919331 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.919337 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-envoy\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:06:06.919331 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.919345 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-credential-socket\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:06:06.919662 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.919355 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8dkj\" (UniqueName: \"kubernetes.io/projected/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-kube-api-access-q8dkj\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:06:06.919662 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.919365 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istio-token\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:06:06.919662 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:06.919375 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268-istiod-ca-cert\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:06:07.255479 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:07.255444 2579 generic.go:356] "Generic (PLEG): container finished" podID="7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" containerID="735ca563ef441d61b7e76899b187cfe0812ee93dc9106d450ea5fbbfdbb4d3b9" exitCode=0 Jul 22 03:06:07.255656 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:07.255506 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" Jul 22 03:06:07.255656 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:07.255523 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" event={"ID":"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268","Type":"ContainerDied","Data":"735ca563ef441d61b7e76899b187cfe0812ee93dc9106d450ea5fbbfdbb4d3b9"} Jul 22 03:06:07.255656 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:07.255603 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s" event={"ID":"7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268","Type":"ContainerDied","Data":"00114b3672ad15d5e79ece3cb3f8aca99796e5f03ffb6171043f89d3eb17ffcc"} Jul 22 03:06:07.255656 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:07.255626 2579 scope.go:117] "RemoveContainer" containerID="735ca563ef441d61b7e76899b187cfe0812ee93dc9106d450ea5fbbfdbb4d3b9" Jul 22 03:06:07.265441 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:07.265422 2579 scope.go:117] "RemoveContainer" containerID="e84ea30d9ae7b89f9e788db578e35e494a7a34c86dfdca02168acf71a7b3897d" Jul 22 03:06:07.273908 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:07.273889 2579 scope.go:117] "RemoveContainer" containerID="735ca563ef441d61b7e76899b187cfe0812ee93dc9106d450ea5fbbfdbb4d3b9" Jul 22 03:06:07.274196 ip-10-0-130-249 kubenswrapper[2579]: E0722 03:06:07.274178 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"735ca563ef441d61b7e76899b187cfe0812ee93dc9106d450ea5fbbfdbb4d3b9\": container with ID starting with 735ca563ef441d61b7e76899b187cfe0812ee93dc9106d450ea5fbbfdbb4d3b9 not found: ID does not exist" containerID="735ca563ef441d61b7e76899b187cfe0812ee93dc9106d450ea5fbbfdbb4d3b9" Jul 22 03:06:07.274243 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:07.274206 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"735ca563ef441d61b7e76899b187cfe0812ee93dc9106d450ea5fbbfdbb4d3b9"} err="failed to get container status \"735ca563ef441d61b7e76899b187cfe0812ee93dc9106d450ea5fbbfdbb4d3b9\": rpc error: code = NotFound desc = could not find container \"735ca563ef441d61b7e76899b187cfe0812ee93dc9106d450ea5fbbfdbb4d3b9\": container with ID starting with 735ca563ef441d61b7e76899b187cfe0812ee93dc9106d450ea5fbbfdbb4d3b9 not found: ID does not exist" Jul 22 03:06:07.274243 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:07.274224 2579 scope.go:117] "RemoveContainer" containerID="e84ea30d9ae7b89f9e788db578e35e494a7a34c86dfdca02168acf71a7b3897d" Jul 22 03:06:07.274499 ip-10-0-130-249 kubenswrapper[2579]: E0722 03:06:07.274482 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e84ea30d9ae7b89f9e788db578e35e494a7a34c86dfdca02168acf71a7b3897d\": container with ID starting with e84ea30d9ae7b89f9e788db578e35e494a7a34c86dfdca02168acf71a7b3897d not found: ID does not exist" containerID="e84ea30d9ae7b89f9e788db578e35e494a7a34c86dfdca02168acf71a7b3897d" Jul 22 03:06:07.274541 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:07.274506 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e84ea30d9ae7b89f9e788db578e35e494a7a34c86dfdca02168acf71a7b3897d"} err="failed to get container status \"e84ea30d9ae7b89f9e788db578e35e494a7a34c86dfdca02168acf71a7b3897d\": rpc error: code = NotFound desc = could not find container \"e84ea30d9ae7b89f9e788db578e35e494a7a34c86dfdca02168acf71a7b3897d\": container with ID starting with e84ea30d9ae7b89f9e788db578e35e494a7a34c86dfdca02168acf71a7b3897d not found: ID does not exist" Jul 22 03:06:07.279159 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:07.279134 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s"] Jul 22 03:06:07.285489 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:07.285460 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-317968-openshift-default-9b567b9df-q7k7s"] Jul 22 03:06:07.628236 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:07.628203 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" path="/var/lib/kubelet/pods/7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268/volumes" Jul 22 03:06:12.502912 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:12.502856 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:16.874289 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:16.874246 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:17.367334 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:17.367291 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:23.912760 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:23.912724 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:24.022406 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:24.021688 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:24.219785 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:24.219695 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:24.440156 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:24.440117 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:24.803280 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:24.803233 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:25.121936 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.121901 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9"] Jul 22 03:06:25.122708 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.122680 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" containerName="istio-proxy" Jul 22 03:06:25.122708 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.122703 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" containerName="istio-proxy" Jul 22 03:06:25.122910 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.122714 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" containerName="istio-proxy" Jul 22 03:06:25.122910 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.122723 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" containerName="istio-proxy" Jul 22 03:06:25.122910 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.122741 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49d57d45-9c9f-4157-adf3-1ac72808ae50" containerName="istio-proxy" Jul 22 03:06:25.122910 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.122749 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="49d57d45-9c9f-4157-adf3-1ac72808ae50" containerName="istio-proxy" Jul 22 03:06:25.122910 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.122868 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" containerName="istio-proxy" Jul 22 03:06:25.122910 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.122886 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c1bd1b0-a9b7-42b3-9ced-7ab1c9adf268" containerName="istio-proxy" Jul 22 03:06:25.127679 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.127658 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.131190 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.131156 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-e35e3d-openshift-default-dockercfg-wfq2f\"" Jul 22 03:06:25.146371 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.146343 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9"] Jul 22 03:06:25.158087 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.158053 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:25.192980 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.192931 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-istio-data\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.192980 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.192980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-credential-socket\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.193202 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.193005 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-istio-envoy\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.193202 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.193084 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dda5458-df53-4889-b30d-4a03dcff761c-istio-token\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.193202 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.193124 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dda5458-df53-4889-b30d-4a03dcff761c-istio-podinfo\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.193202 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.193165 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k276\" (UniqueName: \"kubernetes.io/projected/5dda5458-df53-4889-b30d-4a03dcff761c-kube-api-access-4k276\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.193202 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.193200 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-workload-certs\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.193368 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.193255 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dda5458-df53-4889-b30d-4a03dcff761c-istiod-ca-cert\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.193368 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.193288 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-workload-socket\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.265970 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.265921 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:25.294182 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.294130 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-istio-data\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.294379 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.294193 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-credential-socket\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.294379 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.294226 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-istio-envoy\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.294379 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.294262 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dda5458-df53-4889-b30d-4a03dcff761c-istio-token\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.294379 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.294280 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dda5458-df53-4889-b30d-4a03dcff761c-istio-podinfo\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.294379 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.294321 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k276\" (UniqueName: \"kubernetes.io/projected/5dda5458-df53-4889-b30d-4a03dcff761c-kube-api-access-4k276\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.294379 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.294359 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-workload-certs\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.294705 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.294412 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dda5458-df53-4889-b30d-4a03dcff761c-istiod-ca-cert\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.294705 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.294441 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-workload-socket\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.294705 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.294565 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-istio-data\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.294869 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.294717 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-credential-socket\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.294869 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.294781 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-workload-socket\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.294990 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.294901 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-workload-certs\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.295195 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.295177 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dda5458-df53-4889-b30d-4a03dcff761c-istiod-ca-cert\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.296736 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.296709 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-istio-envoy\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.296902 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.296887 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dda5458-df53-4889-b30d-4a03dcff761c-istio-podinfo\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.307792 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.307769 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k276\" (UniqueName: \"kubernetes.io/projected/5dda5458-df53-4889-b30d-4a03dcff761c-kube-api-access-4k276\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.308092 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.308075 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dda5458-df53-4889-b30d-4a03dcff761c-istio-token\") pod \"e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.446655 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.446566 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:25.610011 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.609980 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9"] Jul 22 03:06:25.611619 ip-10-0-130-249 kubenswrapper[2579]: W0722 03:06:25.611593 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dda5458_df53_4889_b30d_4a03dcff761c.slice/crio-0c76ae8c3ed4f2a8ceef6c80af82573a0a1a08856a481e408990e34adfbfeb43 WatchSource:0}: Error finding container 0c76ae8c3ed4f2a8ceef6c80af82573a0a1a08856a481e408990e34adfbfeb43: Status 404 returned error can't find the container with id 0c76ae8c3ed4f2a8ceef6c80af82573a0a1a08856a481e408990e34adfbfeb43 Jul 22 03:06:25.613813 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.613780 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:06:25.613865 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.613843 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:06:25.613902 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:25.613872 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:06:26.370578 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:26.370541 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" event={"ID":"5dda5458-df53-4889-b30d-4a03dcff761c","Type":"ContainerStarted","Data":"30a17e77243b400abcc2fece7c8ca97bff1c77f615826343f6034609a31d5523"} Jul 22 03:06:26.371011 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:26.370598 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" event={"ID":"5dda5458-df53-4889-b30d-4a03dcff761c","Type":"ContainerStarted","Data":"0c76ae8c3ed4f2a8ceef6c80af82573a0a1a08856a481e408990e34adfbfeb43"} Jul 22 03:06:26.407993 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:26.407921 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" podStartSLOduration=1.4079018859999999 podStartE2EDuration="1.407901886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:06:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:06:26.406920358 +0000 UTC m=+2311.255777203" watchObservedRunningTime="2026-07-22 03:06:26.407901886 +0000 UTC m=+2311.256758719" Jul 22 03:06:26.447097 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:26.447059 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:26.452373 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:26.452343 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:27.376183 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:27.376153 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:27.377362 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:27.377345 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:28.452447 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:28.452406 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:28.626854 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:28.626803 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:28.770364 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:28.770278 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:28.978318 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:28.978287 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:29.817430 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:29.817386 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:29.923452 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:29.923401 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:32.966516 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:32.966477 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:33.449061 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:33.449023 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:33.822200 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:33.822118 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:33.920172 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:33.920136 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:36.811797 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:36.811760 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:36.926036 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:36.926001 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:37.019646 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:37.019607 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:37.379913 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:37.379878 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:37.852325 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:37.852286 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:37.951542 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:37.951506 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:41.353540 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:41.353503 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:42.661450 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:42.661260 2579 patch_prober.go:28] interesting pod/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": read tcp 10.132.0.2:42402->10.132.0.65:15021: read: connection reset by peer" start-of-body= Jul 22 03:06:42.661450 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:42.661360 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" podUID="5dda5458-df53-4889-b30d-4a03dcff761c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": read tcp 10.132.0.2:42402->10.132.0.65:15021: read: connection reset by peer" Jul 22 03:06:43.470310 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:43.470274 2579 generic.go:356] "Generic (PLEG): container finished" podID="5dda5458-df53-4889-b30d-4a03dcff761c" containerID="30a17e77243b400abcc2fece7c8ca97bff1c77f615826343f6034609a31d5523" exitCode=0 Jul 22 03:06:43.470516 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:43.470340 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" event={"ID":"5dda5458-df53-4889-b30d-4a03dcff761c","Type":"ContainerDied","Data":"30a17e77243b400abcc2fece7c8ca97bff1c77f615826343f6034609a31d5523"} Jul 22 03:06:43.470806 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:43.470785 2579 scope.go:117] "RemoveContainer" containerID="30a17e77243b400abcc2fece7c8ca97bff1c77f615826343f6034609a31d5523" Jul 22 03:06:43.471990 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:43.471934 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:06:43.472083 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:43.472061 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:06:43.472124 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:43.472096 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:06:44.481760 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:44.481712 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" event={"ID":"5dda5458-df53-4889-b30d-4a03dcff761c","Type":"ContainerStarted","Data":"fa55879b9f6d4b606fa96dd0ecdf915e537010e6dd19ed3a431bdc6a59a52393"} Jul 22 03:06:45.447127 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:45.447088 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:45.447378 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:45.447357 2579 patch_prober.go:28] interesting pod/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 22 03:06:45.447481 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:45.447454 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" podUID="5dda5458-df53-4889-b30d-4a03dcff761c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 22 03:06:46.447830 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:46.447795 2579 patch_prober.go:28] interesting pod/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 22 03:06:46.448249 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:46.447869 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" podUID="5dda5458-df53-4889-b30d-4a03dcff761c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 22 03:06:47.447180 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:47.447141 2579 patch_prober.go:28] interesting pod/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 22 03:06:47.447370 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:47.447200 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" podUID="5dda5458-df53-4889-b30d-4a03dcff761c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 22 03:06:48.451252 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:48.451214 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:48.451733 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:48.451622 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:48.452433 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:48.452414 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:06:50.195351 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:50.195297 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:50.317161 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:50.317123 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:58.774210 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:58.774172 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:06:58.874635 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:06:58.874600 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:00.295959 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:00.295910 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:00.396789 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:00.396754 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:05.599042 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:05.599005 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:09.612105 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:09.612063 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:10.009541 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:10.009444 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:16.186783 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:16.186744 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9"] Jul 22 03:07:16.187195 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:16.186977 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" podUID="5dda5458-df53-4889-b30d-4a03dcff761c" containerName="istio-proxy" containerID="cri-o://fa55879b9f6d4b606fa96dd0ecdf915e537010e6dd19ed3a431bdc6a59a52393" gracePeriod=30 Jul 22 03:07:16.585706 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:16.585665 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:16.653018 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:16.652984 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:17.171524 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:17.171485 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:17.271910 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:17.271875 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:18.454685 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:18.454641 2579 patch_prober.go:28] interesting pod/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 03:07:18.455095 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:18.454718 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" podUID="5dda5458-df53-4889-b30d-4a03dcff761c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 03:07:21.445275 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.445251 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:07:21.508119 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.508023 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dda5458-df53-4889-b30d-4a03dcff761c-istio-podinfo\") pod \"5dda5458-df53-4889-b30d-4a03dcff761c\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " Jul 22 03:07:21.508119 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.508118 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-istio-envoy\") pod \"5dda5458-df53-4889-b30d-4a03dcff761c\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " Jul 22 03:07:21.508314 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.508144 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-workload-socket\") pod \"5dda5458-df53-4889-b30d-4a03dcff761c\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " Jul 22 03:07:21.508314 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.508194 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-workload-certs\") pod \"5dda5458-df53-4889-b30d-4a03dcff761c\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " Jul 22 03:07:21.508314 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.508238 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dda5458-df53-4889-b30d-4a03dcff761c-istio-token\") pod \"5dda5458-df53-4889-b30d-4a03dcff761c\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " Jul 22 03:07:21.508314 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.508296 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dda5458-df53-4889-b30d-4a03dcff761c-istiod-ca-cert\") pod \"5dda5458-df53-4889-b30d-4a03dcff761c\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " Jul 22 03:07:21.508490 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.508340 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-istio-data\") pod \"5dda5458-df53-4889-b30d-4a03dcff761c\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " Jul 22 03:07:21.508490 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.508369 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-credential-socket\") pod \"5dda5458-df53-4889-b30d-4a03dcff761c\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " Jul 22 03:07:21.508490 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.508438 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4k276\" (UniqueName: \"kubernetes.io/projected/5dda5458-df53-4889-b30d-4a03dcff761c-kube-api-access-4k276\") pod \"5dda5458-df53-4889-b30d-4a03dcff761c\" (UID: \"5dda5458-df53-4889-b30d-4a03dcff761c\") " Jul 22 03:07:21.508647 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.508510 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5dda5458-df53-4889-b30d-4a03dcff761c" (UID: "5dda5458-df53-4889-b30d-4a03dcff761c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:07:21.508734 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.508694 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dda5458-df53-4889-b30d-4a03dcff761c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5dda5458-df53-4889-b30d-4a03dcff761c" (UID: "5dda5458-df53-4889-b30d-4a03dcff761c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:07:21.508804 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.508774 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-workload-socket\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:07:21.508964 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.508926 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5dda5458-df53-4889-b30d-4a03dcff761c" (UID: "5dda5458-df53-4889-b30d-4a03dcff761c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:07:21.509287 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.509217 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5dda5458-df53-4889-b30d-4a03dcff761c" (UID: "5dda5458-df53-4889-b30d-4a03dcff761c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:07:21.509559 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.509513 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-istio-data" (OuterVolumeSpecName: "istio-data") pod "5dda5458-df53-4889-b30d-4a03dcff761c" (UID: "5dda5458-df53-4889-b30d-4a03dcff761c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:07:21.510523 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.510496 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5dda5458-df53-4889-b30d-4a03dcff761c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5dda5458-df53-4889-b30d-4a03dcff761c" (UID: "5dda5458-df53-4889-b30d-4a03dcff761c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:07:21.510523 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.510513 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dda5458-df53-4889-b30d-4a03dcff761c-istio-token" (OuterVolumeSpecName: "istio-token") pod "5dda5458-df53-4889-b30d-4a03dcff761c" (UID: "5dda5458-df53-4889-b30d-4a03dcff761c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:07:21.510650 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.510517 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5dda5458-df53-4889-b30d-4a03dcff761c" (UID: "5dda5458-df53-4889-b30d-4a03dcff761c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:07:21.510785 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.510770 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dda5458-df53-4889-b30d-4a03dcff761c-kube-api-access-4k276" (OuterVolumeSpecName: "kube-api-access-4k276") pod "5dda5458-df53-4889-b30d-4a03dcff761c" (UID: "5dda5458-df53-4889-b30d-4a03dcff761c"). InnerVolumeSpecName "kube-api-access-4k276". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:07:21.609430 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.609388 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dda5458-df53-4889-b30d-4a03dcff761c-istio-podinfo\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:07:21.609430 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.609423 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-istio-envoy\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:07:21.609430 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.609436 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-workload-certs\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:07:21.609678 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.609446 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dda5458-df53-4889-b30d-4a03dcff761c-istio-token\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:07:21.609678 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.609457 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dda5458-df53-4889-b30d-4a03dcff761c-istiod-ca-cert\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:07:21.609678 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.609469 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-istio-data\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:07:21.609678 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.609480 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dda5458-df53-4889-b30d-4a03dcff761c-credential-socket\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:07:21.609678 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.609493 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4k276\" (UniqueName: \"kubernetes.io/projected/5dda5458-df53-4889-b30d-4a03dcff761c-kube-api-access-4k276\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:07:21.684392 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.684353 2579 generic.go:356] "Generic (PLEG): container finished" podID="5dda5458-df53-4889-b30d-4a03dcff761c" containerID="fa55879b9f6d4b606fa96dd0ecdf915e537010e6dd19ed3a431bdc6a59a52393" exitCode=0 Jul 22 03:07:21.684556 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.684398 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" Jul 22 03:07:21.684556 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.684433 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" event={"ID":"5dda5458-df53-4889-b30d-4a03dcff761c","Type":"ContainerDied","Data":"fa55879b9f6d4b606fa96dd0ecdf915e537010e6dd19ed3a431bdc6a59a52393"} Jul 22 03:07:21.684556 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.684489 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9" event={"ID":"5dda5458-df53-4889-b30d-4a03dcff761c","Type":"ContainerDied","Data":"0c76ae8c3ed4f2a8ceef6c80af82573a0a1a08856a481e408990e34adfbfeb43"} Jul 22 03:07:21.684556 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.684512 2579 scope.go:117] "RemoveContainer" containerID="fa55879b9f6d4b606fa96dd0ecdf915e537010e6dd19ed3a431bdc6a59a52393" Jul 22 03:07:21.694112 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.694094 2579 scope.go:117] "RemoveContainer" containerID="30a17e77243b400abcc2fece7c8ca97bff1c77f615826343f6034609a31d5523" Jul 22 03:07:21.704015 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.703985 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9"] Jul 22 03:07:21.705600 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.705174 2579 scope.go:117] "RemoveContainer" containerID="fa55879b9f6d4b606fa96dd0ecdf915e537010e6dd19ed3a431bdc6a59a52393" Jul 22 03:07:21.705816 ip-10-0-130-249 kubenswrapper[2579]: E0722 03:07:21.705777 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa55879b9f6d4b606fa96dd0ecdf915e537010e6dd19ed3a431bdc6a59a52393\": container with ID starting with fa55879b9f6d4b606fa96dd0ecdf915e537010e6dd19ed3a431bdc6a59a52393 not found: ID does not exist" containerID="fa55879b9f6d4b606fa96dd0ecdf915e537010e6dd19ed3a431bdc6a59a52393" Jul 22 03:07:21.706276 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.705837 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa55879b9f6d4b606fa96dd0ecdf915e537010e6dd19ed3a431bdc6a59a52393"} err="failed to get container status \"fa55879b9f6d4b606fa96dd0ecdf915e537010e6dd19ed3a431bdc6a59a52393\": rpc error: code = NotFound desc = could not find container \"fa55879b9f6d4b606fa96dd0ecdf915e537010e6dd19ed3a431bdc6a59a52393\": container with ID starting with fa55879b9f6d4b606fa96dd0ecdf915e537010e6dd19ed3a431bdc6a59a52393 not found: ID does not exist" Jul 22 03:07:21.706276 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.705863 2579 scope.go:117] "RemoveContainer" containerID="30a17e77243b400abcc2fece7c8ca97bff1c77f615826343f6034609a31d5523" Jul 22 03:07:21.706607 ip-10-0-130-249 kubenswrapper[2579]: E0722 03:07:21.706577 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30a17e77243b400abcc2fece7c8ca97bff1c77f615826343f6034609a31d5523\": container with ID starting with 30a17e77243b400abcc2fece7c8ca97bff1c77f615826343f6034609a31d5523 not found: ID does not exist" containerID="30a17e77243b400abcc2fece7c8ca97bff1c77f615826343f6034609a31d5523" Jul 22 03:07:21.706686 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.706603 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30a17e77243b400abcc2fece7c8ca97bff1c77f615826343f6034609a31d5523"} err="failed to get container status \"30a17e77243b400abcc2fece7c8ca97bff1c77f615826343f6034609a31d5523\": rpc error: code = NotFound desc = could not find container \"30a17e77243b400abcc2fece7c8ca97bff1c77f615826343f6034609a31d5523\": container with ID starting with 30a17e77243b400abcc2fece7c8ca97bff1c77f615826343f6034609a31d5523 not found: ID does not exist" Jul 22 03:07:21.707827 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:21.707810 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-e35e3d-openshift-default-7d8d945857-cn2l9"] Jul 22 03:07:23.628923 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:23.628895 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dda5458-df53-4889-b30d-4a03dcff761c" path="/var/lib/kubelet/pods/5dda5458-df53-4889-b30d-4a03dcff761c/volumes" Jul 22 03:07:31.023905 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:31.023870 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:32.069568 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:32.069513 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:36.661302 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:36.661261 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:37.059294 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:37.059197 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:43.060334 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:43.060299 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:43.163764 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:43.163718 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:43.356760 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:43.356719 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:43.570407 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:43.570369 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:43.911594 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:43.911539 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:44.201169 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.201089 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg"] Jul 22 03:07:44.201685 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.201666 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dda5458-df53-4889-b30d-4a03dcff761c" containerName="istio-proxy" Jul 22 03:07:44.201747 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.201690 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dda5458-df53-4889-b30d-4a03dcff761c" containerName="istio-proxy" Jul 22 03:07:44.201747 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.201708 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dda5458-df53-4889-b30d-4a03dcff761c" containerName="istio-proxy" Jul 22 03:07:44.201747 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.201715 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dda5458-df53-4889-b30d-4a03dcff761c" containerName="istio-proxy" Jul 22 03:07:44.201838 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.201826 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dda5458-df53-4889-b30d-4a03dcff761c" containerName="istio-proxy" Jul 22 03:07:44.201874 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.201838 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dda5458-df53-4889-b30d-4a03dcff761c" containerName="istio-proxy" Jul 22 03:07:44.205066 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.205049 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.207725 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.207709 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-643135-openshift-default-dockercfg-hft5g\"" Jul 22 03:07:44.218407 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.218378 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg"] Jul 22 03:07:44.229199 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.229167 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:44.323794 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.323761 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-workload-socket\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.323794 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.323798 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-data\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.324091 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.323833 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-workload-certs\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.324091 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.323924 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-podinfo\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.324091 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.323997 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-envoy\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.324091 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.324022 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd963cac-ca26-4c1a-9648-61c438e4c12a-istiod-ca-cert\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.324091 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.324077 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-credential-socket\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.324324 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.324110 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc2pc\" (UniqueName: \"kubernetes.io/projected/bd963cac-ca26-4c1a-9648-61c438e4c12a-kube-api-access-mc2pc\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.324324 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.324175 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-token\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.424825 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.424786 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-workload-certs\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.425069 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.424854 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-podinfo\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.425069 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.424884 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-envoy\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.425069 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.424905 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd963cac-ca26-4c1a-9648-61c438e4c12a-istiod-ca-cert\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.425243 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.425081 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-credential-socket\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.425243 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.425151 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc2pc\" (UniqueName: \"kubernetes.io/projected/bd963cac-ca26-4c1a-9648-61c438e4c12a-kube-api-access-mc2pc\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.425243 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.425205 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-token\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.425404 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.425260 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-workload-socket\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.425404 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.425278 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-workload-certs\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.425404 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.425291 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-data\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.425559 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.425446 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-credential-socket\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.425559 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.425542 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-data\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.425752 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.425726 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd963cac-ca26-4c1a-9648-61c438e4c12a-istiod-ca-cert\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.425816 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.425775 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-workload-socket\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.427380 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.427356 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-envoy\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.427983 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.427930 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-podinfo\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.441725 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.441702 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-token\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.443090 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.443073 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc2pc\" (UniqueName: \"kubernetes.io/projected/bd963cac-ca26-4c1a-9648-61c438e4c12a-kube-api-access-mc2pc\") pod \"e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.515807 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.515716 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:44.562791 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.562745 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:44.665996 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.665971 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg"] Jul 22 03:07:44.668323 ip-10-0-130-249 kubenswrapper[2579]: W0722 03:07:44.668296 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd963cac_ca26_4c1a_9648_61c438e4c12a.slice/crio-8a66415d89436c42f49219888472e0c5c4bd3230c1bd470d203207802f994418 WatchSource:0}: Error finding container 8a66415d89436c42f49219888472e0c5c4bd3230c1bd470d203207802f994418: Status 404 returned error can't find the container with id 8a66415d89436c42f49219888472e0c5c4bd3230c1bd470d203207802f994418 Jul 22 03:07:44.670452 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.670420 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:07:44.670539 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.670493 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:07:44.670539 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.670524 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:07:44.823461 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.823371 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" event={"ID":"bd963cac-ca26-4c1a-9648-61c438e4c12a","Type":"ContainerStarted","Data":"eeeb870707a6e679e8946d84e2c19c3438e0c4764ec34eb0a5207ec97ee2447a"} Jul 22 03:07:44.823461 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.823427 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" event={"ID":"bd963cac-ca26-4c1a-9648-61c438e4c12a","Type":"ContainerStarted","Data":"8a66415d89436c42f49219888472e0c5c4bd3230c1bd470d203207802f994418"} Jul 22 03:07:44.844274 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:44.843773 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" podStartSLOduration=0.843749903 podStartE2EDuration="843.749903ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:07:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:07:44.843443223 +0000 UTC m=+2389.692300050" watchObservedRunningTime="2026-07-22 03:07:44.843749903 +0000 UTC m=+2389.692606735" Jul 22 03:07:45.516197 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:45.516159 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:45.521081 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:45.521059 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:45.829002 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:45.828972 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:45.829885 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:45.829866 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:07:46.817114 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:46.817079 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:46.917083 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:46.917044 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:47.012618 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:47.012583 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:47.582329 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:47.582282 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:47.704689 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:47.704652 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:47.922860 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:47.922766 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:51.312546 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:51.312508 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:51.645633 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:51.645595 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:52.005246 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:52.005164 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:52.165152 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:52.165106 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:55.149549 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:55.149496 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:55.261084 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:55.261048 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:55.351741 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:55.351687 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:55.450085 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:55.449976 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:55.815024 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:55.814926 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 03:07:55.821957 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:55.821923 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 03:07:55.822435 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:55.822418 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 03:07:55.828584 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:55.828564 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 03:07:55.843362 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:55.843321 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:55.943365 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:55.943317 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:07:59.546751 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:07:59.546709 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:01.871175 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:01.871123 2579 patch_prober.go:28] interesting pod/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 03:08:01.871591 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:01.871208 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" podUID="bd963cac-ca26-4c1a-9648-61c438e4c12a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 03:08:02.930889 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:02.930858 2579 generic.go:356] "Generic (PLEG): container finished" podID="bd963cac-ca26-4c1a-9648-61c438e4c12a" containerID="eeeb870707a6e679e8946d84e2c19c3438e0c4764ec34eb0a5207ec97ee2447a" exitCode=0 Jul 22 03:08:02.931352 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:02.930935 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" event={"ID":"bd963cac-ca26-4c1a-9648-61c438e4c12a","Type":"ContainerDied","Data":"eeeb870707a6e679e8946d84e2c19c3438e0c4764ec34eb0a5207ec97ee2447a"} Jul 22 03:08:02.931399 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:02.931381 2579 scope.go:117] "RemoveContainer" containerID="eeeb870707a6e679e8946d84e2c19c3438e0c4764ec34eb0a5207ec97ee2447a" Jul 22 03:08:02.932444 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:02.932421 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:08:02.932504 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:02.932489 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:08:02.932543 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:02.932519 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 22 03:08:03.939417 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:03.939377 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" event={"ID":"bd963cac-ca26-4c1a-9648-61c438e4c12a","Type":"ContainerStarted","Data":"44af619fb772cbc062db93881cec7f8906be7a58ea14e2490631bcc744a76950"} Jul 22 03:08:04.516991 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:04.516928 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:08:04.517189 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:04.517168 2579 patch_prober.go:28] interesting pod/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 22 03:08:04.517238 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:04.517215 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" podUID="bd963cac-ca26-4c1a-9648-61c438e4c12a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 22 03:08:05.516772 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:05.516733 2579 patch_prober.go:28] interesting pod/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 22 03:08:05.517207 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:05.516794 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" podUID="bd963cac-ca26-4c1a-9648-61c438e4c12a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 22 03:08:06.517249 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:06.517205 2579 patch_prober.go:28] interesting pod/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 22 03:08:06.517654 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:06.517266 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" podUID="bd963cac-ca26-4c1a-9648-61c438e4c12a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 22 03:08:07.516661 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:07.516621 2579 patch_prober.go:28] interesting pod/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 22 03:08:07.516873 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:07.516685 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" podUID="bd963cac-ca26-4c1a-9648-61c438e4c12a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 22 03:08:08.519844 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:08.519815 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:08:08.520275 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:08.520132 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:08:08.520771 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:08.520755 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:08:09.250199 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:09.250164 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:09.454929 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:09.454893 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:09.564278 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:09.564179 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:15.944319 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:15.944284 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:16.046814 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:16.046780 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:17.877864 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:17.877766 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:17.977497 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:17.977465 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:29.027861 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:29.027826 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:30.859466 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:30.859428 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:08:49.508440 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:08:49.508400 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:10:20.426518 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:10:20.426428 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:10:21.833510 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:10:21.833472 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:10:40.105805 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:10:40.105764 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:10:41.002612 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:10:41.002575 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:10:42.626000 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:10:42.625961 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:01.475119 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:11:01.475081 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:07.921203 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:11:07.921171 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:13.515337 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:11:13.515304 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:13.617273 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:11:13.617235 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:44.806421 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:11:44.806384 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:45.627013 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:11:45.626976 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:11:55.519276 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:11:55.519238 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:31.399208 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:12:31.399176 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:12:55.847131 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:12:55.847096 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 03:12:55.853510 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:12:55.853478 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 03:12:55.855065 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:12:55.855050 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 03:12:55.861914 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:12:55.861897 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nl7s9_28495a7f-b408-4ce6-9c4a-555ab7c4c38d/ovn-acl-logging/0.log" Jul 22 03:13:07.857801 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:07.857760 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:08.713615 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:08.713582 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:10.281078 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:10.281036 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:10.382775 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:10.382728 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:16.242110 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:16.242070 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7912-140717351253024"] Jul 22 03:13:16.245497 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:16.245479 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7912-140717351253024" Jul 22 03:13:16.247925 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:16.247906 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-vnmk7\"" Jul 22 03:13:16.251546 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:16.251521 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7912-140717351253024"] Jul 22 03:13:16.335789 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:16.335745 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp4wt\" (UniqueName: \"kubernetes.io/projected/72c7821c-5f34-44c9-b510-122dff9ba708-kube-api-access-bp4wt\") pod \"test-curl-7912-140717351253024\" (UID: \"72c7821c-5f34-44c9-b510-122dff9ba708\") " pod="opendatahub/test-curl-7912-140717351253024" Jul 22 03:13:16.437305 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:16.437265 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp4wt\" (UniqueName: \"kubernetes.io/projected/72c7821c-5f34-44c9-b510-122dff9ba708-kube-api-access-bp4wt\") pod \"test-curl-7912-140717351253024\" (UID: \"72c7821c-5f34-44c9-b510-122dff9ba708\") " pod="opendatahub/test-curl-7912-140717351253024" Jul 22 03:13:16.445796 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:16.445759 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp4wt\" (UniqueName: \"kubernetes.io/projected/72c7821c-5f34-44c9-b510-122dff9ba708-kube-api-access-bp4wt\") pod \"test-curl-7912-140717351253024\" (UID: \"72c7821c-5f34-44c9-b510-122dff9ba708\") " pod="opendatahub/test-curl-7912-140717351253024" Jul 22 03:13:16.556451 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:16.556359 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7912-140717351253024" Jul 22 03:13:16.702380 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:16.702349 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7912-140717351253024"] Jul 22 03:13:16.704851 ip-10-0-130-249 kubenswrapper[2579]: W0722 03:13:16.704825 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72c7821c_5f34_44c9_b510_122dff9ba708.slice/crio-f9a98a18ae2a5321eb278d9a6e12904cbe1de004c7b44cf90eeb45a5c6529274 WatchSource:0}: Error finding container f9a98a18ae2a5321eb278d9a6e12904cbe1de004c7b44cf90eeb45a5c6529274: Status 404 returned error can't find the container with id f9a98a18ae2a5321eb278d9a6e12904cbe1de004c7b44cf90eeb45a5c6529274 Jul 22 03:13:16.706175 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:16.706153 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 03:13:16.743983 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:16.743932 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:16.851448 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:16.851380 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:17.624854 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:17.624810 2579 generic.go:356] "Generic (PLEG): container finished" podID="72c7821c-5f34-44c9-b510-122dff9ba708" containerID="1bdb349fc5d7cdc040e5bb04848583ea817c50d90825715048fd2264f54a8f7d" exitCode=0 Jul 22 03:13:17.628453 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:17.628420 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7912-140717351253024" event={"ID":"72c7821c-5f34-44c9-b510-122dff9ba708","Type":"ContainerDied","Data":"1bdb349fc5d7cdc040e5bb04848583ea817c50d90825715048fd2264f54a8f7d"} Jul 22 03:13:17.628617 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:17.628466 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7912-140717351253024" event={"ID":"72c7821c-5f34-44c9-b510-122dff9ba708","Type":"ContainerStarted","Data":"f9a98a18ae2a5321eb278d9a6e12904cbe1de004c7b44cf90eeb45a5c6529274"} Jul 22 03:13:18.780602 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:18.780575 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7912-140717351253024" Jul 22 03:13:18.802436 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:18.802382 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7912-140717351253024_72c7821c-5f34-44c9-b510-122dff9ba708/test-curl-7912-140717351253024/0.log" Jul 22 03:13:18.827828 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:18.827794 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7912-140717351253024"] Jul 22 03:13:18.833353 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:18.833329 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7912-140717351253024"] Jul 22 03:13:18.835651 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:18.835621 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:18.936299 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:18.936219 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:18.962510 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:18.962473 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bp4wt\" (UniqueName: \"kubernetes.io/projected/72c7821c-5f34-44c9-b510-122dff9ba708-kube-api-access-bp4wt\") pod \"72c7821c-5f34-44c9-b510-122dff9ba708\" (UID: \"72c7821c-5f34-44c9-b510-122dff9ba708\") " Jul 22 03:13:18.964807 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:18.964771 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72c7821c-5f34-44c9-b510-122dff9ba708-kube-api-access-bp4wt" (OuterVolumeSpecName: "kube-api-access-bp4wt") pod "72c7821c-5f34-44c9-b510-122dff9ba708" (UID: "72c7821c-5f34-44c9-b510-122dff9ba708"). InnerVolumeSpecName "kube-api-access-bp4wt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:13:19.064212 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:19.064165 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp4wt\" (UniqueName: \"kubernetes.io/projected/72c7821c-5f34-44c9-b510-122dff9ba708-kube-api-access-bp4wt\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:13:19.628870 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:19.628838 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72c7821c-5f34-44c9-b510-122dff9ba708" path="/var/lib/kubelet/pods/72c7821c-5f34-44c9-b510-122dff9ba708/volumes" Jul 22 03:13:19.636197 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:19.636176 2579 scope.go:117] "RemoveContainer" containerID="1bdb349fc5d7cdc040e5bb04848583ea817c50d90825715048fd2264f54a8f7d" Jul 22 03:13:19.636364 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:19.636207 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7912-140717351253024" Jul 22 03:13:27.799341 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:27.799290 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:27.899249 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:27.899211 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:38.019147 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:38.019108 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:42.349836 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:42.349795 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:42.755074 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:42.754986 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:48.707355 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:48.707312 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:48.807486 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:48.807436 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:48.954922 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:48.954884 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg"] Jul 22 03:13:48.955148 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:48.955123 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" podUID="bd963cac-ca26-4c1a-9648-61c438e4c12a" containerName="istio-proxy" containerID="cri-o://44af619fb772cbc062db93881cec7f8906be7a58ea14e2490631bcc744a76950" gracePeriod=30 Jul 22 03:13:48.978677 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:48.978587 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:49.214586 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:49.214549 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:49.882127 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:49.882093 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:49.985059 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:49.985024 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:13:53.523075 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:53.523037 2579 patch_prober.go:28] interesting pod/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 03:13:53.523459 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:53.523108 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" podUID="bd963cac-ca26-4c1a-9648-61c438e4c12a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 03:13:54.216062 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.215992 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:13:54.293241 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.293207 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-workload-socket\") pod \"bd963cac-ca26-4c1a-9648-61c438e4c12a\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " Jul 22 03:13:54.293431 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.293289 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-credential-socket\") pod \"bd963cac-ca26-4c1a-9648-61c438e4c12a\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " Jul 22 03:13:54.293431 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.293308 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-token\") pod \"bd963cac-ca26-4c1a-9648-61c438e4c12a\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " Jul 22 03:13:54.293431 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.293350 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-data\") pod \"bd963cac-ca26-4c1a-9648-61c438e4c12a\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " Jul 22 03:13:54.293431 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.293370 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mc2pc\" (UniqueName: \"kubernetes.io/projected/bd963cac-ca26-4c1a-9648-61c438e4c12a-kube-api-access-mc2pc\") pod \"bd963cac-ca26-4c1a-9648-61c438e4c12a\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " Jul 22 03:13:54.293431 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.293415 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd963cac-ca26-4c1a-9648-61c438e4c12a-istiod-ca-cert\") pod \"bd963cac-ca26-4c1a-9648-61c438e4c12a\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " Jul 22 03:13:54.293691 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.293456 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-podinfo\") pod \"bd963cac-ca26-4c1a-9648-61c438e4c12a\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " Jul 22 03:13:54.293691 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.293481 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-workload-certs\") pod \"bd963cac-ca26-4c1a-9648-61c438e4c12a\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " Jul 22 03:13:54.293691 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.293508 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-envoy\") pod \"bd963cac-ca26-4c1a-9648-61c438e4c12a\" (UID: \"bd963cac-ca26-4c1a-9648-61c438e4c12a\") " Jul 22 03:13:54.293691 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.293530 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bd963cac-ca26-4c1a-9648-61c438e4c12a" (UID: "bd963cac-ca26-4c1a-9648-61c438e4c12a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:13:54.293691 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.293659 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bd963cac-ca26-4c1a-9648-61c438e4c12a" (UID: "bd963cac-ca26-4c1a-9648-61c438e4c12a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:13:54.293963 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.293811 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-workload-socket\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:13:54.293963 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.293829 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-credential-socket\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:13:54.293963 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.293870 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bd963cac-ca26-4c1a-9648-61c438e4c12a" (UID: "bd963cac-ca26-4c1a-9648-61c438e4c12a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:13:54.293963 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.293915 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd963cac-ca26-4c1a-9648-61c438e4c12a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bd963cac-ca26-4c1a-9648-61c438e4c12a" (UID: "bd963cac-ca26-4c1a-9648-61c438e4c12a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:13:54.294515 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.294420 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-data" (OuterVolumeSpecName: "istio-data") pod "bd963cac-ca26-4c1a-9648-61c438e4c12a" (UID: "bd963cac-ca26-4c1a-9648-61c438e4c12a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:13:54.295692 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.295660 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-token" (OuterVolumeSpecName: "istio-token") pod "bd963cac-ca26-4c1a-9648-61c438e4c12a" (UID: "bd963cac-ca26-4c1a-9648-61c438e4c12a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:13:54.295692 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.295668 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bd963cac-ca26-4c1a-9648-61c438e4c12a" (UID: "bd963cac-ca26-4c1a-9648-61c438e4c12a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:13:54.295862 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.295745 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd963cac-ca26-4c1a-9648-61c438e4c12a-kube-api-access-mc2pc" (OuterVolumeSpecName: "kube-api-access-mc2pc") pod "bd963cac-ca26-4c1a-9648-61c438e4c12a" (UID: "bd963cac-ca26-4c1a-9648-61c438e4c12a"). InnerVolumeSpecName "kube-api-access-mc2pc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:13:54.296043 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.296025 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bd963cac-ca26-4c1a-9648-61c438e4c12a" (UID: "bd963cac-ca26-4c1a-9648-61c438e4c12a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:13:54.394777 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.394739 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd963cac-ca26-4c1a-9648-61c438e4c12a-istiod-ca-cert\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:13:54.394777 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.394769 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-podinfo\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:13:54.394777 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.394777 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-workload-certs\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:13:54.394777 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.394786 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-envoy\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:13:54.395105 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.394793 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-token\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:13:54.395105 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.394800 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd963cac-ca26-4c1a-9648-61c438e4c12a-istio-data\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:13:54.395105 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.394808 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mc2pc\" (UniqueName: \"kubernetes.io/projected/bd963cac-ca26-4c1a-9648-61c438e4c12a-kube-api-access-mc2pc\") on node \"ip-10-0-130-249.ec2.internal\" DevicePath \"\"" Jul 22 03:13:54.827070 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.827036 2579 generic.go:356] "Generic (PLEG): container finished" podID="bd963cac-ca26-4c1a-9648-61c438e4c12a" containerID="44af619fb772cbc062db93881cec7f8906be7a58ea14e2490631bcc744a76950" exitCode=0 Jul 22 03:13:54.827458 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.827088 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" Jul 22 03:13:54.827458 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.827120 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" event={"ID":"bd963cac-ca26-4c1a-9648-61c438e4c12a","Type":"ContainerDied","Data":"44af619fb772cbc062db93881cec7f8906be7a58ea14e2490631bcc744a76950"} Jul 22 03:13:54.827458 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.827173 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg" event={"ID":"bd963cac-ca26-4c1a-9648-61c438e4c12a","Type":"ContainerDied","Data":"8a66415d89436c42f49219888472e0c5c4bd3230c1bd470d203207802f994418"} Jul 22 03:13:54.827458 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.827195 2579 scope.go:117] "RemoveContainer" containerID="44af619fb772cbc062db93881cec7f8906be7a58ea14e2490631bcc744a76950" Jul 22 03:13:54.837137 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.837101 2579 scope.go:117] "RemoveContainer" containerID="eeeb870707a6e679e8946d84e2c19c3438e0c4764ec34eb0a5207ec97ee2447a" Jul 22 03:13:54.845729 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.845712 2579 scope.go:117] "RemoveContainer" containerID="44af619fb772cbc062db93881cec7f8906be7a58ea14e2490631bcc744a76950" Jul 22 03:13:54.846013 ip-10-0-130-249 kubenswrapper[2579]: E0722 03:13:54.845991 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44af619fb772cbc062db93881cec7f8906be7a58ea14e2490631bcc744a76950\": container with ID starting with 44af619fb772cbc062db93881cec7f8906be7a58ea14e2490631bcc744a76950 not found: ID does not exist" containerID="44af619fb772cbc062db93881cec7f8906be7a58ea14e2490631bcc744a76950" Jul 22 03:13:54.846073 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.846024 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44af619fb772cbc062db93881cec7f8906be7a58ea14e2490631bcc744a76950"} err="failed to get container status \"44af619fb772cbc062db93881cec7f8906be7a58ea14e2490631bcc744a76950\": rpc error: code = NotFound desc = could not find container \"44af619fb772cbc062db93881cec7f8906be7a58ea14e2490631bcc744a76950\": container with ID starting with 44af619fb772cbc062db93881cec7f8906be7a58ea14e2490631bcc744a76950 not found: ID does not exist" Jul 22 03:13:54.846073 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.846041 2579 scope.go:117] "RemoveContainer" containerID="eeeb870707a6e679e8946d84e2c19c3438e0c4764ec34eb0a5207ec97ee2447a" Jul 22 03:13:54.846295 ip-10-0-130-249 kubenswrapper[2579]: E0722 03:13:54.846279 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeeb870707a6e679e8946d84e2c19c3438e0c4764ec34eb0a5207ec97ee2447a\": container with ID starting with eeeb870707a6e679e8946d84e2c19c3438e0c4764ec34eb0a5207ec97ee2447a not found: ID does not exist" containerID="eeeb870707a6e679e8946d84e2c19c3438e0c4764ec34eb0a5207ec97ee2447a" Jul 22 03:13:54.846333 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.846302 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeeb870707a6e679e8946d84e2c19c3438e0c4764ec34eb0a5207ec97ee2447a"} err="failed to get container status \"eeeb870707a6e679e8946d84e2c19c3438e0c4764ec34eb0a5207ec97ee2447a\": rpc error: code = NotFound desc = could not find container \"eeeb870707a6e679e8946d84e2c19c3438e0c4764ec34eb0a5207ec97ee2447a\": container with ID starting with eeeb870707a6e679e8946d84e2c19c3438e0c4764ec34eb0a5207ec97ee2447a not found: ID does not exist" Jul 22 03:13:54.849329 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.849305 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg"] Jul 22 03:13:54.855205 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:54.855183 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-643135-openshift-default-8665b8688c-m2wqg"] Jul 22 03:13:55.629483 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:13:55.629452 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd963cac-ca26-4c1a-9648-61c438e4c12a" path="/var/lib/kubelet/pods/bd963cac-ca26-4c1a-9648-61c438e4c12a/volumes" Jul 22 03:14:00.086258 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:00.086219 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:04.433484 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:04.433443 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:04.882695 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:04.882662 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:11.021380 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:11.021339 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:11.127165 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:11.127131 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:11.305262 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:11.305165 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:11.539186 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:11.539149 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:17.533760 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:17.533668 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-zvl82_08508ac3-8c86-4698-901a-9d4da42e2296/manager/0.log" Jul 22 03:14:17.907418 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:17.907393 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-qh54n_1db31f22-8184-45f8-9293-c6d7a7c3a80c/server/0.log" Jul 22 03:14:18.035606 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:18.035574 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gr7fk_3b03853b-c8fb-4fd2-a4d8-26e836673c79/manager/0.log" Jul 22 03:14:18.152452 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:18.152421 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86c79b8765-msg5n_9eee98d0-955c-4c42-8b41-c5a7af22abbc/copy-manifests/0.log" Jul 22 03:14:18.172094 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:18.171984 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86c79b8765-msg5n_9eee98d0-955c-4c42-8b41-c5a7af22abbc/manager/0.log" Jul 22 03:14:18.893197 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:18.893166 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744805-55t8g_02f5ca2b-6521-47e5-b53b-9d3436fcb2af/cleanup/0.log" Jul 22 03:14:20.601686 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:20.601645 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-cxftt_4b2e3ecd-4e6d-4e2c-8b69-9c38e2f1e24c/manager/0.log" Jul 22 03:14:20.722853 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:20.722820 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-wz94h_74fc1862-c7a1-4cef-93ef-3a5e4407687a/manager/0.log" Jul 22 03:14:20.966958 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:20.966861 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4bs7r_47f8f702-2641-4227-a18b-0845dc3e7e47/registry-server/0.log" Jul 22 03:14:21.096483 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:21.096449 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jrhd7_89089245-cb77-4847-b496-fc24592a732b/manager/0.log" Jul 22 03:14:21.351629 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:21.351598 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-wnn6s_8a6156f2-efc7-44cb-9f2b-8a42aa13e712/manager/0.log" Jul 22 03:14:21.942304 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:21.942273 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7fd555b8c5-6slpl_83709665-5bae-491c-b361-b0bbf7dba1f4/kube-auth-proxy/0.log" Jul 22 03:14:22.182968 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:22.182907 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-flz22_53ddb162-08b8-45cd-b2ed-d3557318eb6f/istio-proxy/1.log" Jul 22 03:14:22.306064 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:22.305980 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-zzvzx_a0d891a3-b1ac-468b-b4de-bf3faa5ad570/payload-pre-processing/0.log" Jul 22 03:14:22.551169 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:22.551138 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5889846976-rpxs8_5beca28a-e7a9-410d-ae0f-dce53b6faeae/router/0.log" Jul 22 03:14:22.909798 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:22.909755 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-kb9vl_aa51ef7e-691c-4919-92dc-e0eb24ed4b77/main/0.log" Jul 22 03:14:23.026358 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:23.026332 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-dn244_60327b17-f4d5-4837-b9df-27c4f7b16ba5/main/0.log" Jul 22 03:14:23.147133 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:23.147098 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-4psmq_3bbcd8b7-6e16-442f-9e6c-69b365e18b85/main/0.log" Jul 22 03:14:29.434403 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:29.434361 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:29.533200 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:29.533168 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" limit=1024 Jul 22 03:14:32.395927 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:32.395897 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cjr5p_72d84003-0a34-4663-be5f-09d9d7a14498/global-pull-secret-syncer/0.log" Jul 22 03:14:32.496311 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:32.496276 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-j2l7h_b76d49de-e528-431e-bb2a-710c095b1779/konnectivity-agent/0.log" Jul 22 03:14:32.623140 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:32.623107 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-249.ec2.internal_046dd7cb569e08833c71642bc20f3e71/haproxy/0.log" Jul 22 03:14:37.938173 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:37.938144 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-cxftt_4b2e3ecd-4e6d-4e2c-8b69-9c38e2f1e24c/manager/0.log" Jul 22 03:14:37.976516 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:37.976484 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-wz94h_74fc1862-c7a1-4cef-93ef-3a5e4407687a/manager/0.log" Jul 22 03:14:38.068030 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:38.068001 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4bs7r_47f8f702-2641-4227-a18b-0845dc3e7e47/registry-server/0.log" Jul 22 03:14:38.313934 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:38.313835 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jrhd7_89089245-cb77-4847-b496-fc24592a732b/manager/0.log" Jul 22 03:14:38.439837 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:38.439803 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-wnn6s_8a6156f2-efc7-44cb-9f2b-8a42aa13e712/manager/0.log" Jul 22 03:14:40.416496 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:40.416456 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-85lxl_9f11a91e-5064-4838-bd58-7ab3369cc834/cluster-monitoring-operator/0.log" Jul 22 03:14:40.606352 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:40.606323 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2s67d_b92cff58-42df-4c45-88f4-b3e5bc97f9aa/node-exporter/0.log" Jul 22 03:14:40.629864 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:40.629839 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2s67d_b92cff58-42df-4c45-88f4-b3e5bc97f9aa/kube-rbac-proxy/0.log" Jul 22 03:14:40.652099 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:40.652073 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2s67d_b92cff58-42df-4c45-88f4-b3e5bc97f9aa/init-textfile/0.log" Jul 22 03:14:41.282454 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:41.282427 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-7nf8q_fddc14b7-ef5f-497b-af06-a15a0e952656/prometheus-operator-admission-webhook/0.log" Jul 22 03:14:42.662013 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.661969 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-66nws_f1da4aa5-c459-4bfd-9a72-5e179d8af2e3/networking-console-plugin/0.log" Jul 22 03:14:42.782156 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.782121 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95"] Jul 22 03:14:42.782553 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.782541 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd963cac-ca26-4c1a-9648-61c438e4c12a" containerName="istio-proxy" Jul 22 03:14:42.782599 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.782555 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd963cac-ca26-4c1a-9648-61c438e4c12a" containerName="istio-proxy" Jul 22 03:14:42.782599 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.782571 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd963cac-ca26-4c1a-9648-61c438e4c12a" containerName="istio-proxy" Jul 22 03:14:42.782599 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.782576 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd963cac-ca26-4c1a-9648-61c438e4c12a" containerName="istio-proxy" Jul 22 03:14:42.782599 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.782587 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72c7821c-5f34-44c9-b510-122dff9ba708" containerName="test-curl-7912-140717351253024" Jul 22 03:14:42.782599 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.782593 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="72c7821c-5f34-44c9-b510-122dff9ba708" containerName="test-curl-7912-140717351253024" Jul 22 03:14:42.782750 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.782655 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd963cac-ca26-4c1a-9648-61c438e4c12a" containerName="istio-proxy" Jul 22 03:14:42.782750 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.782664 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="72c7821c-5f34-44c9-b510-122dff9ba708" containerName="test-curl-7912-140717351253024" Jul 22 03:14:42.785774 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.785758 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:42.788267 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.788244 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gt8kb\"/\"kube-root-ca.crt\"" Jul 22 03:14:42.789149 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.789116 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gt8kb\"/\"default-dockercfg-sxttz\"" Jul 22 03:14:42.789268 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.789131 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gt8kb\"/\"openshift-service-ca.crt\"" Jul 22 03:14:42.794675 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.794652 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95"] Jul 22 03:14:42.949462 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.949371 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b7679340-d5a0-4da0-83e4-2d2060d2565b-podres\") pod \"perf-node-gather-daemonset-mzq95\" (UID: \"b7679340-d5a0-4da0-83e4-2d2060d2565b\") " pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:42.949462 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.949416 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v2ck\" (UniqueName: \"kubernetes.io/projected/b7679340-d5a0-4da0-83e4-2d2060d2565b-kube-api-access-9v2ck\") pod \"perf-node-gather-daemonset-mzq95\" (UID: \"b7679340-d5a0-4da0-83e4-2d2060d2565b\") " pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:42.949651 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.949514 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b7679340-d5a0-4da0-83e4-2d2060d2565b-lib-modules\") pod \"perf-node-gather-daemonset-mzq95\" (UID: \"b7679340-d5a0-4da0-83e4-2d2060d2565b\") " pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:42.949651 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.949555 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7679340-d5a0-4da0-83e4-2d2060d2565b-sys\") pod \"perf-node-gather-daemonset-mzq95\" (UID: \"b7679340-d5a0-4da0-83e4-2d2060d2565b\") " pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:42.949651 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:42.949574 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b7679340-d5a0-4da0-83e4-2d2060d2565b-proc\") pod \"perf-node-gather-daemonset-mzq95\" (UID: \"b7679340-d5a0-4da0-83e4-2d2060d2565b\") " pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:43.050887 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:43.050846 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b7679340-d5a0-4da0-83e4-2d2060d2565b-lib-modules\") pod \"perf-node-gather-daemonset-mzq95\" (UID: \"b7679340-d5a0-4da0-83e4-2d2060d2565b\") " pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:43.051088 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:43.050897 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7679340-d5a0-4da0-83e4-2d2060d2565b-sys\") pod \"perf-node-gather-daemonset-mzq95\" (UID: \"b7679340-d5a0-4da0-83e4-2d2060d2565b\") " pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:43.051088 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:43.051001 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7679340-d5a0-4da0-83e4-2d2060d2565b-sys\") pod \"perf-node-gather-daemonset-mzq95\" (UID: \"b7679340-d5a0-4da0-83e4-2d2060d2565b\") " pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:43.051088 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:43.051009 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b7679340-d5a0-4da0-83e4-2d2060d2565b-proc\") pod \"perf-node-gather-daemonset-mzq95\" (UID: \"b7679340-d5a0-4da0-83e4-2d2060d2565b\") " pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:43.051088 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:43.051048 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b7679340-d5a0-4da0-83e4-2d2060d2565b-proc\") pod \"perf-node-gather-daemonset-mzq95\" (UID: \"b7679340-d5a0-4da0-83e4-2d2060d2565b\") " pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:43.051088 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:43.051065 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b7679340-d5a0-4da0-83e4-2d2060d2565b-lib-modules\") pod \"perf-node-gather-daemonset-mzq95\" (UID: \"b7679340-d5a0-4da0-83e4-2d2060d2565b\") " pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:43.051088 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:43.051089 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b7679340-d5a0-4da0-83e4-2d2060d2565b-podres\") pod \"perf-node-gather-daemonset-mzq95\" (UID: \"b7679340-d5a0-4da0-83e4-2d2060d2565b\") " pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:43.051280 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:43.051113 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v2ck\" (UniqueName: \"kubernetes.io/projected/b7679340-d5a0-4da0-83e4-2d2060d2565b-kube-api-access-9v2ck\") pod \"perf-node-gather-daemonset-mzq95\" (UID: \"b7679340-d5a0-4da0-83e4-2d2060d2565b\") " pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:43.051280 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:43.051189 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b7679340-d5a0-4da0-83e4-2d2060d2565b-podres\") pod \"perf-node-gather-daemonset-mzq95\" (UID: \"b7679340-d5a0-4da0-83e4-2d2060d2565b\") " pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:43.059451 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:43.059430 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v2ck\" (UniqueName: \"kubernetes.io/projected/b7679340-d5a0-4da0-83e4-2d2060d2565b-kube-api-access-9v2ck\") pod \"perf-node-gather-daemonset-mzq95\" (UID: \"b7679340-d5a0-4da0-83e4-2d2060d2565b\") " pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:43.096470 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:43.096440 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:43.241064 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:43.240851 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95"] Jul 22 03:14:43.244269 ip-10-0-130-249 kubenswrapper[2579]: W0722 03:14:43.244237 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb7679340_d5a0_4da0_83e4_2d2060d2565b.slice/crio-eb2d0ecdc17cf35af5b8738019f86f7c38bc913f83ed059c32fd2062ca80c7eb WatchSource:0}: Error finding container eb2d0ecdc17cf35af5b8738019f86f7c38bc913f83ed059c32fd2062ca80c7eb: Status 404 returned error can't find the container with id eb2d0ecdc17cf35af5b8738019f86f7c38bc913f83ed059c32fd2062ca80c7eb Jul 22 03:14:43.313719 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:43.313689 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/1.log" Jul 22 03:14:43.322384 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:43.322354 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4fgl4_d4f98d41-0f79-4730-a09a-844c18aef0d4/console-operator/2.log" Jul 22 03:14:43.808935 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:43.808910 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-f4df9d7cf-x5ccv_a0edd0e0-9dbc-4052-a2cc-f87a81fb0782/console/0.log" Jul 22 03:14:43.852696 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:43.852660 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-phhmt_2500f8f3-355d-47ef-af40-da1b1e9a6ccd/download-server/0.log" Jul 22 03:14:44.091770 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:44.091741 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" event={"ID":"b7679340-d5a0-4da0-83e4-2d2060d2565b","Type":"ContainerStarted","Data":"63ac33d2a56c6ab5dead85d3a511482cf69e220d27b47f7fb3f19b67a793f052"} Jul 22 03:14:44.091992 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:44.091788 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" event={"ID":"b7679340-d5a0-4da0-83e4-2d2060d2565b","Type":"ContainerStarted","Data":"eb2d0ecdc17cf35af5b8738019f86f7c38bc913f83ed059c32fd2062ca80c7eb"} Jul 22 03:14:44.091992 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:44.091829 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:44.110878 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:44.110825 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" podStartSLOduration=2.110809972 podStartE2EDuration="2.110809972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:14:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:14:44.1082421 +0000 UTC m=+2808.957098931" watchObservedRunningTime="2026-07-22 03:14:44.110809972 +0000 UTC m=+2808.959666805" Jul 22 03:14:44.458154 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:44.458071 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-qwwcl_1d65fb9e-9e26-42f4-958d-67e85c7ddc31/volume-data-source-validator/0.log" Jul 22 03:14:45.719481 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:45.719451 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6gfmw_7243e221-39df-4cbe-bd32-302e9744977d/dns/0.log" Jul 22 03:14:45.765233 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:45.765197 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6gfmw_7243e221-39df-4cbe-bd32-302e9744977d/kube-rbac-proxy/0.log" Jul 22 03:14:46.076825 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:46.076803 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vzq9c_21af75d3-4d8f-49b7-b96f-db2dc2cd5275/dns-node-resolver/0.log" Jul 22 03:14:46.660257 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:46.660227 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fwbqh_7369bc37-a20a-44e7-9af0-2b15ac190fce/node-ca/0.log" Jul 22 03:14:47.874214 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:47.874165 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7fd555b8c5-6slpl_83709665-5bae-491c-b361-b0bbf7dba1f4/kube-auth-proxy/0.log" Jul 22 03:14:47.935425 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:47.935366 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-flz22_53ddb162-08b8-45cd-b2ed-d3557318eb6f/istio-proxy/0.log" Jul 22 03:14:47.986303 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:47.986265 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-flz22_53ddb162-08b8-45cd-b2ed-d3557318eb6f/istio-proxy/1.log" Jul 22 03:14:48.029466 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:48.029431 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-zzvzx_a0d891a3-b1ac-468b-b4de-bf3faa5ad570/payload-pre-processing/0.log" Jul 22 03:14:48.093546 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:48.093513 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5889846976-rpxs8_5beca28a-e7a9-410d-ae0f-dce53b6faeae/router/0.log" Jul 22 03:14:48.902563 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:48.902532 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-x2pkg_4a41203b-ba6d-4c90-a657-9c4cd9a748e9/serve-healthcheck-canary/0.log" Jul 22 03:14:49.393990 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:49.393957 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-tq2tv_858fe6a9-2e1c-48c7-8e67-12119323253e/insights-operator/0.log" Jul 22 03:14:49.395592 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:49.395554 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-tq2tv_858fe6a9-2e1c-48c7-8e67-12119323253e/insights-operator/1.log" Jul 22 03:14:49.424551 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:49.424524 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8dnzm_7f7165bc-e726-4334-8e57-49c05f994762/kube-rbac-proxy/0.log" Jul 22 03:14:49.451016 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:49.450990 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8dnzm_7f7165bc-e726-4334-8e57-49c05f994762/exporter/0.log" Jul 22 03:14:49.478868 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:49.478841 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8dnzm_7f7165bc-e726-4334-8e57-49c05f994762/extractor/0.log" Jul 22 03:14:50.108468 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:50.108438 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gt8kb/perf-node-gather-daemonset-mzq95" Jul 22 03:14:51.856408 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:51.856363 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-zvl82_08508ac3-8c86-4698-901a-9d4da42e2296/manager/0.log" Jul 22 03:14:52.263752 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:52.263670 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-qh54n_1db31f22-8184-45f8-9293-c6d7a7c3a80c/server/0.log" Jul 22 03:14:52.374356 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:52.374321 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gr7fk_3b03853b-c8fb-4fd2-a4d8-26e836673c79/manager/0.log" Jul 22 03:14:52.489115 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:52.489081 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86c79b8765-msg5n_9eee98d0-955c-4c42-8b41-c5a7af22abbc/manager/0.log" Jul 22 03:14:52.511984 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:52.511932 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86c79b8765-msg5n_9eee98d0-955c-4c42-8b41-c5a7af22abbc/copy-manifests/0.log" Jul 22 03:14:53.578816 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:53.578783 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7fd555b8c5-6slpl_83709665-5bae-491c-b361-b0bbf7dba1f4/kube-auth-proxy/0.log" Jul 22 03:14:53.617816 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:53.617783 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-flz22_53ddb162-08b8-45cd-b2ed-d3557318eb6f/istio-proxy/0.log" Jul 22 03:14:53.672053 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:53.671975 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-flz22_53ddb162-08b8-45cd-b2ed-d3557318eb6f/istio-proxy/1.log" Jul 22 03:14:53.701065 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:53.701033 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-zzvzx_a0d891a3-b1ac-468b-b4de-bf3faa5ad570/payload-pre-processing/0.log" Jul 22 03:14:53.747334 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:53.747299 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5889846976-rpxs8_5beca28a-e7a9-410d-ae0f-dce53b6faeae/router/0.log" Jul 22 03:14:55.079286 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:55.079254 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7fd555b8c5-6slpl_83709665-5bae-491c-b361-b0bbf7dba1f4/kube-auth-proxy/0.log" Jul 22 03:14:55.116676 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:55.116648 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-flz22_53ddb162-08b8-45cd-b2ed-d3557318eb6f/istio-proxy/0.log" Jul 22 03:14:55.169190 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:55.169157 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-flz22_53ddb162-08b8-45cd-b2ed-d3557318eb6f/istio-proxy/1.log" Jul 22 03:14:55.200543 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:55.200507 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-zzvzx_a0d891a3-b1ac-468b-b4de-bf3faa5ad570/payload-pre-processing/0.log" Jul 22 03:14:55.247262 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:55.247234 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5889846976-rpxs8_5beca28a-e7a9-410d-ae0f-dce53b6faeae/router/0.log" Jul 22 03:14:59.043735 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:59.043702 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-v44rl_a68a9d1b-bcef-479b-a21d-45eaca9c7a40/migrator/0.log" Jul 22 03:14:59.064420 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:59.064391 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-v44rl_a68a9d1b-bcef-479b-a21d-45eaca9c7a40/graceful-termination/0.log" Jul 22 03:14:59.432807 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:59.432770 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-924jv_3bf725ad-6951-48c4-b2e4-0d898eec59ed/kube-storage-version-migrator-operator/1.log" Jul 22 03:14:59.434387 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:14:59.434361 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-924jv_3bf725ad-6951-48c4-b2e4-0d898eec59ed/kube-storage-version-migrator-operator/0.log" Jul 22 03:15:00.708396 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:15:00.708366 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wpncq_d18b98e3-c43e-44ff-8425-f19881254719/kube-multus-additional-cni-plugins/0.log" Jul 22 03:15:00.739236 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:15:00.739203 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wpncq_d18b98e3-c43e-44ff-8425-f19881254719/egress-router-binary-copy/0.log" Jul 22 03:15:00.767350 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:15:00.767314 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wpncq_d18b98e3-c43e-44ff-8425-f19881254719/cni-plugins/0.log" Jul 22 03:15:00.795912 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:15:00.795878 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wpncq_d18b98e3-c43e-44ff-8425-f19881254719/bond-cni-plugin/0.log" Jul 22 03:15:00.821994 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:15:00.821935 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wpncq_d18b98e3-c43e-44ff-8425-f19881254719/routeoverride-cni/0.log" Jul 22 03:15:00.852075 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:15:00.852039 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wpncq_d18b98e3-c43e-44ff-8425-f19881254719/whereabouts-cni-bincopy/0.log" Jul 22 03:15:00.878653 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:15:00.878616 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wpncq_d18b98e3-c43e-44ff-8425-f19881254719/whereabouts-cni/0.log" Jul 22 03:15:01.104086 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:15:01.104049 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-cs678_f7d6dd73-d6a9-499a-acab-235e3a23d126/kube-multus/0.log" Jul 22 03:15:01.166671 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:15:01.166641 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-48qrg_ddc83329-034a-42bd-a8b3-9bf395c15e9b/network-metrics-daemon/0.log" Jul 22 03:15:01.197126 ip-10-0-130-249 kubenswrapper[2579]: I0722 03:15:01.197098 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-48qrg_ddc83329-034a-42bd-a8b3-9bf395c15e9b/kube-rbac-proxy/0.log"