Jul 08 18:15:42.874494 ip-10-0-134-97 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 18:15:42.874513 ip-10-0-134-97 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 18:15:42.874523 ip-10-0-134-97 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 18:15:42.874847 ip-10-0-134-97 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 18:15:52.958484 ip-10-0-134-97 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 18:15:52.958499 ip-10-0-134-97 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a8ee61e058aa4a73843ff62fd074f7dd -- Jul 08 18:17:57.290260 ip-10-0-134-97 systemd[1]: Starting Kubernetes Kubelet... Jul 08 18:17:57.677007 ip-10-0-134-97 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:17:57.677007 ip-10-0-134-97 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 18:17:57.677007 ip-10-0-134-97 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:17:57.677007 ip-10-0-134-97 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 18:17:57.677007 ip-10-0-134-97 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:17:57.679470 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.679381 2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 18:17:57.683869 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683854 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:17:57.683869 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683869 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683873 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683876 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683879 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683882 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683884 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683887 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683890 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683892 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683895 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683898 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683900 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683903 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683905 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683908 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683916 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683918 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683921 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683924 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683926 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:17:57.683934 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683929 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683931 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683934 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683936 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683952 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683955 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683958 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683961 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683963 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683966 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683968 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683971 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683974 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683976 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683979 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683981 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683984 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683988 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683992 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:17:57.684419 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683995 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.683998 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684001 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684004 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684006 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684009 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684012 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684014 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684017 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684019 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684021 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684024 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684027 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684030 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684032 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684035 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684038 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684041 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684044 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684046 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:17:57.684866 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684048 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684051 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684055 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684058 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684061 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684063 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684065 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684068 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684070 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684073 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684075 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684078 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684082 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684086 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684089 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684091 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684094 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684097 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684101 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:17:57.685364 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684104 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684107 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684109 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684112 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684115 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684117 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684120 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684123 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684126 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684129 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684131 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684134 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684136 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684139 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684142 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684144 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684147 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684149 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684152 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684154 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:17:57.685847 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684841 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684847 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684850 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684853 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684855 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684858 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684861 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684863 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684866 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684868 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684871 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684873 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684876 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684879 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684884 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684888 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684890 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684893 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684896 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:17:57.686350 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684900 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684904 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684907 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684909 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684912 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684915 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684917 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684920 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684922 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684925 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684927 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684930 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684932 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684935 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684954 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684957 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684959 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684963 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684966 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684969 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:17:57.686808 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684971 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684974 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684977 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684981 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684984 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684987 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684990 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684993 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684995 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.684998 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685001 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685004 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685006 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685009 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685012 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685014 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685017 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685019 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685022 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:17:57.687302 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685024 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685027 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685030 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685032 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685034 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685037 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685039 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685042 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685044 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685047 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685049 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685052 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685054 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685058 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685061 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685063 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685066 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685068 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685072 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685074 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:17:57.687765 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685077 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685080 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685082 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685085 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685088 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685090 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685093 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685095 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685098 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685100 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685103 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685105 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685108 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685110 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685112 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685115 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685117 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685120 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685122 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685125 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:17:57.688283 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685127 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685484 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685489 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685492 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685494 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685498 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685501 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685503 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685506 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685508 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685511 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685514 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685516 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685519 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685521 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685524 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685527 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685529 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685532 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:17:57.688760 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685534 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685537 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685539 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685542 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685544 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685547 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685549 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685554 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685557 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685559 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685563 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685567 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685569 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685572 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685576 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685580 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685582 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685585 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685588 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:17:57.689230 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685591 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685593 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685596 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685598 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685601 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685604 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685607 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685609 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685612 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685615 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685617 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685620 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685622 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685624 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685627 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685629 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685632 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685634 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685637 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685639 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:17:57.689755 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685643 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685646 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685648 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685651 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685653 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685656 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685658 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685661 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685663 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685665 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685668 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685670 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685673 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685675 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685678 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685680 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685683 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685685 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685688 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685690 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:17:57.690259 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685692 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685695 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685697 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685700 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685702 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685705 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685708 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685710 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685713 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685715 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685718 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685720 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685724 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685726 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685729 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685731 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685734 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685736 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685738 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685741 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:17:57.690745 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685743 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.685746 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685818 2574 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685825 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685831 2574 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685835 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685839 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685845 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685851 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685858 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685863 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685868 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685873 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685878 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685881 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685884 2574 flags.go:64] FLAG: --cgroup-root="" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685887 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685890 2574 flags.go:64] FLAG: --client-ca-file="" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685892 2574 flags.go:64] FLAG: --cloud-provider="external" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685895 2574 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685900 2574 flags.go:64] FLAG: --cluster-domain="" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685903 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685906 2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685909 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 18:17:57.691280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685913 2574 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685916 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685920 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685922 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685926 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685928 2574 flags.go:64] FLAG: --contention-profiling="false" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685931 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685934 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685952 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685956 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685963 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685967 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685972 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685976 2574 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685978 2574 flags.go:64] FLAG: --enable-server="true" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685982 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685986 2574 flags.go:64] FLAG: --event-burst="100" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685989 2574 flags.go:64] FLAG: --event-qps="50" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685992 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685995 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.685997 2574 flags.go:64] FLAG: --eviction-hard="" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686002 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686005 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686008 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686011 2574 flags.go:64] FLAG: --eviction-soft="" Jul 08 18:17:57.691848 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686013 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686016 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686019 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686022 2574 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686025 2574 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686027 2574 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686030 2574 flags.go:64] FLAG: --feature-gates="" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686037 2574 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686040 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686043 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686046 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686048 2574 flags.go:64] FLAG: --healthz-port="10248" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686051 2574 flags.go:64] FLAG: --help="false" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686054 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-134-97.ec2.internal" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686057 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686060 2574 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686062 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686065 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686069 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686071 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686074 2574 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686076 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686080 2574 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 18:17:57.692480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686083 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686086 2574 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686089 2574 flags.go:64] FLAG: --kube-reserved="" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686091 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686094 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686098 2574 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686101 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686103 2574 flags.go:64] FLAG: --lock-file="" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686106 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686108 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686111 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686116 2574 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686119 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686122 2574 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686124 2574 flags.go:64] FLAG: --logging-format="text" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686127 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686131 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686133 2574 flags.go:64] FLAG: --manifest-url="" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686136 2574 flags.go:64] FLAG: --manifest-url-header="" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686140 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686143 2574 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686147 2574 flags.go:64] FLAG: --max-pods="110" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686150 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686153 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686155 2574 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 18:17:57.693167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686158 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686161 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686164 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686166 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686173 2574 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686176 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686179 2574 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686182 2574 flags.go:64] FLAG: --pod-cidr="" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686185 2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686190 2574 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686193 2574 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686196 2574 flags.go:64] FLAG: --pods-per-core="0" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686199 2574 flags.go:64] FLAG: --port="10250" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686203 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686206 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f894c1e1a71b9792" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686209 2574 flags.go:64] FLAG: --qos-reserved="" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686211 2574 flags.go:64] FLAG: --read-only-port="10255" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686214 2574 flags.go:64] FLAG: --register-node="true" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686217 2574 flags.go:64] FLAG: --register-with-taints="" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686220 2574 flags.go:64] FLAG: --registry-burst="10" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686223 2574 flags.go:64] FLAG: --registry-qps="5" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686226 2574 flags.go:64] FLAG: --reserved-cpus="" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686229 2574 flags.go:64] FLAG: --reserved-memory="" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686232 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686235 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 18:17:57.693761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686238 2574 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686240 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686243 2574 flags.go:64] FLAG: --runonce="false" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686246 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686248 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686251 2574 flags.go:64] FLAG: --seccomp-default="false" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686254 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686256 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686259 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686262 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686265 2574 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686267 2574 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686270 2574 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686272 2574 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686275 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686278 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686281 2574 flags.go:64] FLAG: --system-cgroups="" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686284 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686289 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686291 2574 flags.go:64] FLAG: --tls-cert-file="" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686295 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686300 2574 flags.go:64] FLAG: --tls-min-version="" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686302 2574 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686305 2574 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686308 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 18:17:57.694381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686310 2574 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686313 2574 flags.go:64] FLAG: --v="2" Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686317 2574 flags.go:64] FLAG: --version="false" Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686321 2574 flags.go:64] FLAG: --vmodule="" Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686324 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686328 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686438 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686442 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686445 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686449 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686453 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686456 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686459 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686461 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686464 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686466 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686469 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686472 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686474 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686476 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686479 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:17:57.695027 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686481 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686484 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686486 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686489 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686492 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686494 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686497 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686500 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686502 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686505 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686507 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686509 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686512 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686514 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686517 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686519 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686523 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686526 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686528 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686530 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:17:57.695547 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686533 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686535 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686538 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686540 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686543 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686546 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686549 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686552 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686554 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686556 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686560 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686564 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686567 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686569 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686572 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686574 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686577 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686580 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686585 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:17:57.696124 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686587 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686590 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686592 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686595 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686597 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686599 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686602 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686604 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686607 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686611 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686613 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686616 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686618 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686621 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686623 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686626 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686629 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686631 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686633 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686636 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:17:57.696595 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686639 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686641 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686643 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686646 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686648 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686651 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686655 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686658 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686661 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686664 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686667 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686671 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686673 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686676 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686678 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686680 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686683 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686685 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686688 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:17:57.697107 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686690 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:17:57.697575 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686693 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:17:57.697575 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686697 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:17:57.697575 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686699 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:17:57.697575 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686701 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:17:57.697575 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.686704 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:17:57.697575 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.686709 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:17:57.697575 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.693352 2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 18:17:57.697575 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.693367 2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 18:17:57.697575 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.694115 2574 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 18:17:57.697575 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.694122 2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 18:17:57.697575 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694179 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:17:57.697575 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694185 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:17:57.697575 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694188 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:17:57.697575 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694191 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:17:57.697575 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694195 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694198 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694201 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694205 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694209 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694212 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694215 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694218 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694221 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694225 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694229 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694232 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694235 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694237 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694240 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694242 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694245 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694247 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694249 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:17:57.697962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694252 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694254 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694257 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694259 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694262 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694264 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694268 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694271 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694274 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694276 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694279 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694282 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694284 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694287 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694289 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694291 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694294 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694296 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694299 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694301 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:17:57.698417 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694303 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694306 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694309 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694312 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694314 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694317 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694319 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694322 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694324 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694326 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694329 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694331 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694333 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694336 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694338 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694341 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694343 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694346 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694350 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694352 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:17:57.698898 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694355 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694357 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694359 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694362 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694365 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694367 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694369 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694372 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694374 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694376 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694379 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694381 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694385 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694389 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694392 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694396 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694398 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694401 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694404 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:17:57.699401 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694406 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:17:57.699862 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694409 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:17:57.699862 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694411 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:17:57.699862 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694414 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:17:57.699862 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694417 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:17:57.699862 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694419 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:17:57.699862 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694422 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:17:57.699862 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694424 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:17:57.699862 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694426 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:17:57.699862 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694429 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:17:57.699862 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694431 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:17:57.699862 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694434 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:17:57.699862 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694437 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:17:57.699862 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694439 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:17:57.699862 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694441 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:17:57.699862 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694444 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:17:57.699862 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694447 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:17:57.700347 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.694451 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:17:57.700347 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694559 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:17:57.700347 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694564 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:17:57.700347 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694568 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:17:57.700347 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694571 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:17:57.700347 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694574 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:17:57.700347 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694577 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:17:57.700347 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694579 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:17:57.700347 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694582 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:17:57.700347 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694584 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:17:57.700347 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694587 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:17:57.700347 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694589 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:17:57.700347 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694592 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:17:57.700347 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694595 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694597 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694600 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694602 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694605 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694607 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694610 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694612 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694615 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694617 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694620 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694622 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694624 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694628 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694630 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694632 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694635 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694637 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694640 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694642 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:17:57.700695 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694644 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694647 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694649 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694651 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694654 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694657 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694659 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694662 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694664 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694667 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694669 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694671 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694674 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694677 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694679 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694681 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694684 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694686 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694689 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:17:57.701190 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694691 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694695 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694698 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694701 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694703 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694706 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694709 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694712 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694714 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694717 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694719 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694722 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694725 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694728 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694731 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694733 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694736 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694739 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694741 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:17:57.701800 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694743 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694746 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694748 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694750 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694753 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694755 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694758 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694760 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694763 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694765 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694767 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694770 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694772 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694775 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694777 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694779 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694782 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694784 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694786 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694789 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:17:57.702478 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694791 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:17:57.703093 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694793 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:17:57.703093 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694795 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:17:57.703093 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694798 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:17:57.703093 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694800 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:17:57.703093 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694803 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:17:57.703093 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694805 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:17:57.703093 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694807 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:17:57.703093 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:57.694810 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:17:57.703093 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.694815 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:17:57.703093 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.694908 2574 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 18:17:57.703093 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.696739 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 18:17:57.703093 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.697988 2574 server.go:1019] "Starting client certificate rotation" Jul 08 18:17:57.703093 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.698086 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 18:17:57.703093 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.698116 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 18:17:57.703476 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.700523 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 18:17:57.704225 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.704185 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 18:17:57.714205 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.714187 2574 log.go:25] "Validated CRI v1 runtime API" Jul 08 18:17:57.716662 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.716644 2574 log.go:25] "Validated CRI v1 image API" Jul 08 18:17:57.717410 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.717394 2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 18:17:57.719878 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.719848 2574 fs.go:135] Filesystem UUIDs: map[27a3c1b8-817f-43eb-88ac-e17ee36aab39:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 bc2bacf3-22d5-4401-b54e-c96335ca5643:/dev/nvme0n1p3] Jul 08 18:17:57.719970 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.719878 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 18:17:57.724922 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.724812 2574 manager.go:216] Machine: {Timestamp:2026-07-08 18:17:57.723489497 +0000 UTC m=+0.313706199 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096469 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b0048b3aa6ba9355f8a9c98ce30ab SystemUUID:ec2b0048-b3aa-6ba9-355f-8a9c98ce30ab BootID:a8ee61e0-58aa-4a73-843f-f62fd074f7dd Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3b:78:90:1e:b9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3b:78:90:1e:b9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1a:da:50:60:f0:49 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 18:17:57.724922 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.724917 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 18:17:57.725039 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.725008 2574 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 18:17:57.727768 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.727743 2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 18:17:57.727936 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.727771 2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-97.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 18:17:57.728502 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.728489 2574 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 18:17:57.728585 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.728509 2574 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 18:17:57.728585 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.728520 2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 18:17:57.729107 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.729095 2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 18:17:57.730221 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.730209 2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 18:17:57.732764 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.732750 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 08 18:17:57.732890 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.732879 2574 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 18:17:57.733624 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.733614 2574 kubelet.go:486] "Attempting to sync node with API server" Jul 08 18:17:57.733683 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.733632 2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 18:17:57.733683 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.733646 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 18:17:57.733683 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.733660 2574 kubelet.go:398] "Adding apiserver pod source" Jul 08 18:17:57.733683 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.733672 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 18:17:57.734250 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.734226 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 18:17:57.734320 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.734267 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 18:17:57.736310 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.736295 2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 18:17:57.737997 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.737982 2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 18:17:57.738072 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.738019 2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 18:17:57.739000 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.738980 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 18:17:57.739000 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.739000 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 18:17:57.739106 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.739009 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 18:17:57.739106 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.739017 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 18:17:57.739106 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.739030 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 18:17:57.739106 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.739075 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 18:17:57.739106 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.739095 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 18:17:57.739106 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.739103 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 18:17:57.739312 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.739112 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 18:17:57.739312 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.739121 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 18:17:57.739312 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.739141 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 18:17:57.739312 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.739149 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 18:17:57.739926 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.739913 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 18:17:57.740015 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.739930 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 18:17:57.741771 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.741740 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 18:17:57.743845 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.743824 2574 server.go:1268] "Started kubelet" Jul 08 18:17:57.744834 ip-10-0-134-97 systemd[1]: Started Kubernetes Kubelet. Jul 08 18:17:57.745433 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.745283 2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 18:17:57.745562 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.745417 2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 18:17:57.745616 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.745599 2574 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 18:17:57.746290 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.746269 2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-97.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 18:17:57.746384 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:57.746299 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 18:17:57.746497 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:57.746281 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-97.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 18:17:57.746674 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.746656 2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 18:17:57.748099 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.748085 2574 server.go:310] "Adding debug handlers to kubelet server" Jul 08 18:17:57.750809 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:57.750791 2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 18:17:57.752262 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.752247 2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 18:17:57.752262 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.752253 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 18:17:57.752423 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:57.752388 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-97.ec2.internal\" not found" Jul 08 18:17:57.752792 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.752778 2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 18:17:57.752888 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.752780 2574 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 18:17:57.752888 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.752889 2574 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 18:17:57.753080 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.752989 2574 reconstruct.go:97] "Volume reconstruction finished" Jul 08 18:17:57.753080 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.752999 2574 reconciler.go:29] "Reconciler: start to sync state" Jul 08 18:17:57.753310 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.753293 2574 factory.go:55] Registering systemd factory Jul 08 18:17:57.753376 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.753342 2574 factory.go:223] Registration of the systemd container factory successfully Jul 08 18:17:57.753583 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.753567 2574 factory.go:153] Registering CRI-O factory Jul 08 18:17:57.753583 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.753586 2574 factory.go:223] Registration of the crio container factory successfully Jul 08 18:17:57.753699 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.753635 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 18:17:57.753699 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.753657 2574 factory.go:103] Registering Raw factory Jul 08 18:17:57.753699 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.753684 2574 manager.go:1196] Started watching for new ooms in manager Jul 08 18:17:57.754162 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.754148 2574 manager.go:349] Starting recovery of all containers Jul 08 18:17:57.754274 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:57.753438 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-97.ec2.internal.18c06356ff823225 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-97.ec2.internal,UID:ip-10-0-134-97.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-97.ec2.internal,},FirstTimestamp:2026-07-08 18:17:57.743776293 +0000 UTC m=+0.333992999,LastTimestamp:2026-07-08 18:17:57.743776293 +0000 UTC m=+0.333992999,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-97.ec2.internal,}" Jul 08 18:17:57.763920 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.763814 2574 manager.go:354] Recovery completed Jul 08 18:17:57.766038 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:57.765927 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 18:17:57.766142 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:57.766055 2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-97.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 18:17:57.768091 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.768079 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:17:57.769860 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.769843 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:17:57.769917 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.769873 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:17:57.769917 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.769883 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:17:57.770390 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.770374 2574 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 18:17:57.770390 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.770385 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:17:57.770506 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.770388 2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 18:17:57.770506 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.770442 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 08 18:17:57.771198 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.771182 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:17:57.771272 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.771208 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:17:57.771272 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.771228 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:17:57.771463 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.771447 2574 policy_none.go:49] "None policy: Start" Jul 08 18:17:57.771499 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.771470 2574 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 18:17:57.771499 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.771485 2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 18:17:57.772334 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.772321 2574 policy_none.go:47] "Start" Jul 08 18:17:57.773517 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.773501 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s2skb" Jul 08 18:17:57.778888 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.778868 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s2skb" Jul 08 18:17:57.780754 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:57.780689 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-97.ec2.internal.18c0635701103654 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-97.ec2.internal,UID:ip-10-0-134-97.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-97.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-97.ec2.internal,},FirstTimestamp:2026-07-08 18:17:57.769860692 +0000 UTC m=+0.360077393,LastTimestamp:2026-07-08 18:17:57.769860692 +0000 UTC m=+0.360077393,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-97.ec2.internal,}" Jul 08 18:17:57.808519 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.808503 2574 manager.go:337] "Starting Device Plugin manager" Jul 08 18:17:57.808632 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:57.808541 2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 18:17:57.808632 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.808555 2574 server.go:87] "Starting device plugin registration server" Jul 08 18:17:57.808810 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.808797 2574 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 18:17:57.808869 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.808811 2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 18:17:57.808921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.808902 2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 18:17:57.809023 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.809006 2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 18:17:57.809023 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.809020 2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 18:17:57.809545 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:57.809524 2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 18:17:57.809621 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:57.809581 2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-97.ec2.internal\" not found" Jul 08 18:17:57.909346 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.909317 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:17:57.911280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.911259 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:17:57.911382 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.911293 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:17:57.911382 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.911304 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:17:57.911382 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.911331 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-97.ec2.internal" Jul 08 18:17:57.919323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.919302 2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-97.ec2.internal" Jul 08 18:17:57.919323 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:57.919325 2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-97.ec2.internal\": node \"ip-10-0-134-97.ec2.internal\" not found" Jul 08 18:17:57.923742 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.923709 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 18:17:57.925268 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.925251 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 18:17:57.925342 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.925292 2574 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 18:17:57.925342 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.925312 2574 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 18:17:57.925415 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:57.925348 2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 18:17:57.927597 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:57.927548 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:17:57.934488 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:57.934472 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-97.ec2.internal\" not found" Jul 08 18:17:58.026400 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.026375 2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-97.ec2.internal"] Jul 08 18:17:58.026516 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.026474 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:17:58.028239 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.028224 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:17:58.028311 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.028252 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:17:58.028311 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.028261 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:17:58.029455 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.029443 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:17:58.029587 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.029571 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal" Jul 08 18:17:58.029621 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.029605 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:17:58.030347 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.030331 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:17:58.030397 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.030363 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:17:58.030397 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.030373 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:17:58.030468 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.030324 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:17:58.030468 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.030441 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:17:58.030468 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.030454 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:17:58.031563 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.031547 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-97.ec2.internal" Jul 08 18:17:58.031633 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.031574 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:17:58.032312 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.032294 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:17:58.032396 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.032324 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:17:58.032396 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.032335 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:17:58.034812 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:58.034797 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-97.ec2.internal\" not found" Jul 08 18:17:58.054393 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.054373 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/069618fd9935d30ef59a0acde000b9cb-config\") pod \"kube-apiserver-proxy-ip-10-0-134-97.ec2.internal\" (UID: \"069618fd9935d30ef59a0acde000b9cb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-97.ec2.internal" Jul 08 18:17:58.054464 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.054396 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c38e96b9a88058f800f8db7067621905-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal\" (UID: \"c38e96b9a88058f800f8db7067621905\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal" Jul 08 18:17:58.054464 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.054413 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c38e96b9a88058f800f8db7067621905-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal\" (UID: \"c38e96b9a88058f800f8db7067621905\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal" Jul 08 18:17:58.060502 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:58.060485 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-97.ec2.internal\" not found" node="ip-10-0-134-97.ec2.internal" Jul 08 18:17:58.064713 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:58.064697 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-97.ec2.internal\" not found" node="ip-10-0-134-97.ec2.internal" Jul 08 18:17:58.135711 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:58.135689 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-97.ec2.internal\" not found" Jul 08 18:17:58.155521 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.155495 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c38e96b9a88058f800f8db7067621905-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal\" (UID: \"c38e96b9a88058f800f8db7067621905\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal" Jul 08 18:17:58.155580 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.155527 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c38e96b9a88058f800f8db7067621905-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal\" (UID: \"c38e96b9a88058f800f8db7067621905\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal" Jul 08 18:17:58.155580 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.155543 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/069618fd9935d30ef59a0acde000b9cb-config\") pod \"kube-apiserver-proxy-ip-10-0-134-97.ec2.internal\" (UID: \"069618fd9935d30ef59a0acde000b9cb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-97.ec2.internal" Jul 08 18:17:58.155642 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.155593 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/069618fd9935d30ef59a0acde000b9cb-config\") pod \"kube-apiserver-proxy-ip-10-0-134-97.ec2.internal\" (UID: \"069618fd9935d30ef59a0acde000b9cb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-97.ec2.internal" Jul 08 18:17:58.155642 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.155601 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c38e96b9a88058f800f8db7067621905-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal\" (UID: \"c38e96b9a88058f800f8db7067621905\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal" Jul 08 18:17:58.155642 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.155602 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c38e96b9a88058f800f8db7067621905-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal\" (UID: \"c38e96b9a88058f800f8db7067621905\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal" Jul 08 18:17:58.235905 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:58.235829 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-97.ec2.internal\" not found" Jul 08 18:17:58.336492 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:58.336459 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-97.ec2.internal\" not found" Jul 08 18:17:58.363037 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.363012 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal" Jul 08 18:17:58.367600 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.367586 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-97.ec2.internal" Jul 08 18:17:58.436901 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:58.436859 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-97.ec2.internal\" not found" Jul 08 18:17:58.537433 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:58.537348 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-97.ec2.internal\" not found" Jul 08 18:17:58.637888 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:58.637859 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-97.ec2.internal\" not found" Jul 08 18:17:58.697465 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.697432 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 18:17:58.698000 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.697563 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 18:17:58.738906 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:58.738857 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-97.ec2.internal\" not found" Jul 08 18:17:58.780959 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.780901 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 18:12:59 +0000 UTC" deadline="2028-03-24 01:33:17.135962304 +0000 UTC" Jul 08 18:17:58.780959 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.780951 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14983h15m18.355027695s" Jul 08 18:17:58.839279 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:17:58.839226 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-97.ec2.internal\" not found" Jul 08 18:17:58.872695 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.872669 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:17:58.875175 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.875154 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:17:58.875275 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.875189 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:17:58.875275 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.875202 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:17:58.876917 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.876902 2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:17:58.936857 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:58.936822 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod069618fd9935d30ef59a0acde000b9cb.slice/crio-4e405ea9b50fc760c21e4b6d91e52510b43fb57de90b5e2770b750c9df9d5918 WatchSource:0}: Error finding container 4e405ea9b50fc760c21e4b6d91e52510b43fb57de90b5e2770b750c9df9d5918: Status 404 returned error can't find the container with id 4e405ea9b50fc760c21e4b6d91e52510b43fb57de90b5e2770b750c9df9d5918 Jul 08 18:17:58.937300 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:17:58.937279 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc38e96b9a88058f800f8db7067621905.slice/crio-b2a9cf2dd41bb335e9f402eecc4d00e6abfd75aba51d36fc05f534ca508434b2 WatchSource:0}: Error finding container b2a9cf2dd41bb335e9f402eecc4d00e6abfd75aba51d36fc05f534ca508434b2: Status 404 returned error can't find the container with id b2a9cf2dd41bb335e9f402eecc4d00e6abfd75aba51d36fc05f534ca508434b2 Jul 08 18:17:58.941336 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.941321 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:17:58.953018 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.952998 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-97.ec2.internal" Jul 08 18:17:58.962531 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.962514 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 18:17:58.964288 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:17:58.964272 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal" Jul 08 18:18:00.812538 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:00.812523 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 18:18:00.834789 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:00.834770 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:18:01.175998 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.175924 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:18:01.573749 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.573657 2574 apiserver.go:52] "Watching apiserver" Jul 08 18:18:01.580697 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.580680 2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 18:18:01.581078 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.581058 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-8svs8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4","openshift-multus/multus-additional-cni-plugins-tmdn8","openshift-multus/multus-dkrkt","openshift-multus/network-metrics-daemon-ts8sd","openshift-network-diagnostics/network-check-target-fq9lp","kube-system/global-pull-secret-syncer-xw9f2","kube-system/kube-apiserver-proxy-ip-10-0-134-97.ec2.internal","openshift-cluster-node-tuning-operator/tuned-hst6g","openshift-dns/node-resolver-rxdnb","openshift-image-registry/node-ca-smjlq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal","openshift-network-operator/iptables-alerter-pt8xd","openshift-ovn-kubernetes/ovnkube-node-x56pf"] Jul 08 18:18:01.582446 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.582431 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:01.582507 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:01.582494 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xw9f2" podUID="506c4790-d442-4464-8b83-06d2891a6df0" Jul 08 18:18:01.583322 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.583309 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pt8xd" Jul 08 18:18:01.584266 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.584251 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.585631 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.585611 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 18:18:01.585954 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.585932 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:18:01.586643 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.586390 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.586643 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.586504 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-f5bks\"" Jul 08 18:18:01.587136 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.586980 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 18:18:01.587136 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.586968 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 18:18:01.587136 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.586999 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 18:18:01.587378 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.587261 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 18:18:01.587378 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.587264 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-r6xvx\"" Jul 08 18:18:01.587378 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.587293 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 18:18:01.587530 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.587394 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 18:18:01.587786 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.587770 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:01.587868 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:01.587828 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:01.588598 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.588581 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 18:18:01.588688 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.588604 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5jpc2\"" Jul 08 18:18:01.588952 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.588924 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:01.589042 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:01.588983 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:01.590125 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.590109 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.591335 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.591204 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8svs8" Jul 08 18:18:01.592456 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.592435 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:18:01.592549 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.592496 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 18:18:01.592549 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.592535 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mwshw\"" Jul 08 18:18:01.592661 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.592646 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 18:18:01.595151 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.594057 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 18:18:01.595151 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.594231 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cwg49\"" Jul 08 18:18:01.595151 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.594306 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 18:18:01.595806 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.595786 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.596885 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.596869 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rxdnb" Jul 08 18:18:01.597765 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.597746 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 18:18:01.597867 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.597789 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-smjlq" Jul 08 18:18:01.598012 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.597993 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 18:18:01.598069 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.598000 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 18:18:01.598260 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.598245 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tszg6\"" Jul 08 18:18:01.599112 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.599086 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.599260 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.599243 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 18:18:01.599437 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.599413 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tk92d\"" Jul 08 18:18:01.599508 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.599484 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 18:18:01.600188 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.600174 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 18:18:01.600277 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.600241 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mpbz6\"" Jul 08 18:18:01.600277 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.600250 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 18:18:01.600277 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.600265 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 18:18:01.600975 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.600960 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 18:18:01.601341 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.601326 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 18:18:01.601473 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.601458 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 18:18:01.602280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602260 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7zdck\"" Jul 08 18:18:01.602365 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602321 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 18:18:01.602365 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602322 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 18:18:01.602553 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602535 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/506c4790-d442-4464-8b83-06d2891a6df0-kubelet-config\") pod \"global-pull-secret-syncer-xw9f2\" (UID: \"506c4790-d442-4464-8b83-06d2891a6df0\") " pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:01.602609 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602569 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-hostroot\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.602609 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602601 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33915cb2-f6ca-4540-9425-ff653a28c8a2-system-cni-dir\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.602700 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602628 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a4e7ee8d-935d-4f12-a691-c123e9a5fd27-agent-certs\") pod \"konnectivity-agent-8svs8\" (UID: \"a4e7ee8d-935d-4f12-a691-c123e9a5fd27\") " pod="kube-system/konnectivity-agent-8svs8" Jul 08 18:18:01.602700 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602652 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a4e7ee8d-935d-4f12-a691-c123e9a5fd27-konnectivity-ca\") pod \"konnectivity-agent-8svs8\" (UID: \"a4e7ee8d-935d-4f12-a691-c123e9a5fd27\") " pod="kube-system/konnectivity-agent-8svs8" Jul 08 18:18:01.602700 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602677 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-sys-fs\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.602842 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602701 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkjrm\" (UniqueName: \"kubernetes.io/projected/f3fd4161-4fee-43ef-ad4a-e784880b494a-kube-api-access-bkjrm\") pod \"iptables-alerter-pt8xd\" (UID: \"f3fd4161-4fee-43ef-ad4a-e784880b494a\") " pod="openshift-network-operator/iptables-alerter-pt8xd" Jul 08 18:18:01.602842 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602726 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs\") pod \"network-metrics-daemon-ts8sd\" (UID: \"0d868453-b10b-4dd9-b843-91de3f1b48d3\") " pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:01.602842 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602770 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-multus-conf-dir\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.602842 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602801 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-tmp\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.602842 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602823 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-os-release\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.603092 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602862 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ff6bcc41-a870-4794-96f3-418b31e9a666-multus-daemon-config\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.603092 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602898 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpbgx\" (UniqueName: \"kubernetes.io/projected/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-kube-api-access-lpbgx\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.603092 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602915 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 18:18:01.603092 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602920 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/33915cb2-f6ca-4540-9425-ff653a28c8a2-os-release\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.603092 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602942 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-modprobe-d\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.603092 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602963 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-run\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.603092 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.602987 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret\") pod \"global-pull-secret-syncer-xw9f2\" (UID: \"506c4790-d442-4464-8b83-06d2891a6df0\") " pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:01.603092 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603008 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-etc-selinux\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.603092 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603062 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/506c4790-d442-4464-8b83-06d2891a6df0-dbus\") pod \"global-pull-secret-syncer-xw9f2\" (UID: \"506c4790-d442-4464-8b83-06d2891a6df0\") " pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:01.603092 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603085 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-run-multus-certs\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.603489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603114 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt9bc\" (UniqueName: \"kubernetes.io/projected/ff6bcc41-a870-4794-96f3-418b31e9a666-kube-api-access-xt9bc\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.603489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603165 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/33915cb2-f6ca-4540-9425-ff653a28c8a2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.603489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603218 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-var-lib-cni-bin\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.603489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603243 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdg8p\" (UniqueName: \"kubernetes.io/projected/33915cb2-f6ca-4540-9425-ff653a28c8a2-kube-api-access-bdg8p\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.603489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603269 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.603489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603292 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-run-netns\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.603489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603315 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-var-lib-kubelet\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.603489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603360 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-sysctl-conf\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.603489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603383 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-sys\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.603489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603405 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/33915cb2-f6ca-4540-9425-ff653a28c8a2-cnibin\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.603489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603427 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ff6bcc41-a870-4794-96f3-418b31e9a666-cni-binary-copy\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.603489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603454 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-tuned\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.603489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603485 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/33915cb2-f6ca-4540-9425-ff653a28c8a2-cni-binary-copy\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.603910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603506 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/33915cb2-f6ca-4540-9425-ff653a28c8a2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.603910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603530 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-socket-dir\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.603910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603550 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-device-dir\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.603910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603564 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shjkn\" (UniqueName: \"kubernetes.io/projected/aaf14a01-c937-4cd9-aa30-e1de026e5680-kube-api-access-shjkn\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.603910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603600 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-host\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.603910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603628 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-var-lib-kubelet\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.603910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603649 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-multus-socket-dir-parent\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.603910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603666 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-run-k8s-cni-cncf-io\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.603910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603685 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-kubernetes\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.603910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603707 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-systemd\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.603910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603739 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f3fd4161-4fee-43ef-ad4a-e784880b494a-iptables-alerter-script\") pod \"iptables-alerter-pt8xd\" (UID: \"f3fd4161-4fee-43ef-ad4a-e784880b494a\") " pod="openshift-network-operator/iptables-alerter-pt8xd" Jul 08 18:18:01.603910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603757 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-system-cni-dir\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.603910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603775 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-multus-cni-dir\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.603910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603800 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-sysconfig\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.603910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603818 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3fd4161-4fee-43ef-ad4a-e784880b494a-host-slash\") pod \"iptables-alerter-pt8xd\" (UID: \"f3fd4161-4fee-43ef-ad4a-e784880b494a\") " pod="openshift-network-operator/iptables-alerter-pt8xd" Jul 08 18:18:01.603910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603834 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-registration-dir\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.604640 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603878 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-cnibin\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.604640 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603903 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g5wf\" (UniqueName: \"kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf\") pod \"network-check-target-fq9lp\" (UID: \"3a8d7a1d-49b0-41a0-a297-2898090ea58f\") " pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:01.604640 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603936 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-lib-modules\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.604640 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603964 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/33915cb2-f6ca-4540-9425-ff653a28c8a2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.604640 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.603985 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwn8k\" (UniqueName: \"kubernetes.io/projected/0d868453-b10b-4dd9-b843-91de3f1b48d3-kube-api-access-qwn8k\") pod \"network-metrics-daemon-ts8sd\" (UID: \"0d868453-b10b-4dd9-b843-91de3f1b48d3\") " pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:01.604640 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.604010 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-var-lib-cni-multus\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.604640 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.604051 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-etc-kubernetes\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.604640 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.604071 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-sysctl-d\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.610242 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.610225 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 18:18:01.632038 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.632016 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ttvkw" Jul 08 18:18:01.639362 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.639342 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ttvkw" Jul 08 18:18:01.692616 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.692585 2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 18:18:01.704922 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.704889 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdg8p\" (UniqueName: \"kubernetes.io/projected/33915cb2-f6ca-4540-9425-ff653a28c8a2-kube-api-access-bdg8p\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.705082 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.704935 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.705082 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705016 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.705082 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705047 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-run-netns\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.705082 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705063 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-var-lib-kubelet\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.705082 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705080 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-sysctl-conf\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.705334 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705117 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-var-lib-kubelet\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.705334 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705140 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-sys\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.705334 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705148 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-run-netns\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.705334 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705168 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-kubelet\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.705334 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705206 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-var-lib-openvswitch\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.705334 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705219 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-sys\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.705334 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705261 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/33915cb2-f6ca-4540-9425-ff653a28c8a2-cnibin\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.705334 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705282 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ff6bcc41-a870-4794-96f3-418b31e9a666-cni-binary-copy\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.705334 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705303 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-tuned\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.705334 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705304 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/33915cb2-f6ca-4540-9425-ff653a28c8a2-cnibin\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.705334 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705326 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-etc-openvswitch\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.705334 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705337 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-sysctl-conf\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.705821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705351 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-run-openvswitch\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.705821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705379 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-cni-netd\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.705821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705399 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a05de7da-b5fb-4f8b-b52f-d9222471fae2-env-overrides\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.705821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705418 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dfd1b4d7-5544-40c0-af2a-f22db9888c91-hosts-file\") pod \"node-resolver-rxdnb\" (UID: \"dfd1b4d7-5544-40c0-af2a-f22db9888c91\") " pod="openshift-dns/node-resolver-rxdnb" Jul 08 18:18:01.705821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705439 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/33915cb2-f6ca-4540-9425-ff653a28c8a2-cni-binary-copy\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.705821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705468 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/33915cb2-f6ca-4540-9425-ff653a28c8a2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.705821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705548 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-socket-dir\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.705821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705573 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-device-dir\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.705821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705597 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shjkn\" (UniqueName: \"kubernetes.io/projected/aaf14a01-c937-4cd9-aa30-e1de026e5680-kube-api-access-shjkn\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.705821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705623 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-host\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.705821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705637 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/33915cb2-f6ca-4540-9425-ff653a28c8a2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.705821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705647 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-node-log\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.705821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705670 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-var-lib-kubelet\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.705821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705694 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-multus-socket-dir-parent\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.705821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705709 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-socket-dir\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.705821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705717 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-run-k8s-cni-cncf-io\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.705821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705756 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-run-k8s-cni-cncf-io\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705767 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-kubernetes\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705797 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-systemd\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705809 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-host\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705821 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f3fd4161-4fee-43ef-ad4a-e784880b494a-iptables-alerter-script\") pod \"iptables-alerter-pt8xd\" (UID: \"f3fd4161-4fee-43ef-ad4a-e784880b494a\") " pod="openshift-network-operator/iptables-alerter-pt8xd" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705876 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-run-ovn-kubernetes\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705888 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-var-lib-kubelet\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705909 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-system-cni-dir\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705934 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-multus-cni-dir\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705908 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-device-dir\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705936 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-multus-socket-dir-parent\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705960 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-sysconfig\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705978 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-kubernetes\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705979 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/33915cb2-f6ca-4540-9425-ff653a28c8a2-cni-binary-copy\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705935 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ff6bcc41-a870-4794-96f3-418b31e9a666-cni-binary-copy\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.705984 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3fd4161-4fee-43ef-ad4a-e784880b494a-host-slash\") pod \"iptables-alerter-pt8xd\" (UID: \"f3fd4161-4fee-43ef-ad4a-e784880b494a\") " pod="openshift-network-operator/iptables-alerter-pt8xd" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706010 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-system-cni-dir\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706024 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3fd4161-4fee-43ef-ad4a-e784880b494a-host-slash\") pod \"iptables-alerter-pt8xd\" (UID: \"f3fd4161-4fee-43ef-ad4a-e784880b494a\") " pod="openshift-network-operator/iptables-alerter-pt8xd" Jul 08 18:18:01.706323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706053 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-sysconfig\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.706972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706058 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-systemd\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.706972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706085 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjpxm\" (UniqueName: \"kubernetes.io/projected/ec2217c0-db7c-4660-a6f3-990b90cbf03d-kube-api-access-rjpxm\") pod \"node-ca-smjlq\" (UID: \"ec2217c0-db7c-4660-a6f3-990b90cbf03d\") " pod="openshift-image-registry/node-ca-smjlq" Jul 08 18:18:01.706972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706097 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-multus-cni-dir\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.706972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706113 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-registration-dir\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.706972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706133 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-cnibin\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.706972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706149 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g5wf\" (UniqueName: \"kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf\") pod \"network-check-target-fq9lp\" (UID: \"3a8d7a1d-49b0-41a0-a297-2898090ea58f\") " pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:01.706972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706151 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-registration-dir\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.706972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706164 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-lib-modules\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.706972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706179 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/33915cb2-f6ca-4540-9425-ff653a28c8a2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.706972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706196 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwn8k\" (UniqueName: \"kubernetes.io/projected/0d868453-b10b-4dd9-b843-91de3f1b48d3-kube-api-access-qwn8k\") pod \"network-metrics-daemon-ts8sd\" (UID: \"0d868453-b10b-4dd9-b843-91de3f1b48d3\") " pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:01.706972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706207 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-cnibin\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.706972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706238 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-var-lib-cni-multus\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.706972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706210 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-var-lib-cni-multus\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.706972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706368 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-lib-modules\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.706972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706388 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-etc-kubernetes\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.706972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706397 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f3fd4161-4fee-43ef-ad4a-e784880b494a-iptables-alerter-script\") pod \"iptables-alerter-pt8xd\" (UID: \"f3fd4161-4fee-43ef-ad4a-e784880b494a\") " pod="openshift-network-operator/iptables-alerter-pt8xd" Jul 08 18:18:01.706972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706416 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-sysctl-d\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.707629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706442 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-log-socket\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.707629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706467 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnxmh\" (UniqueName: \"kubernetes.io/projected/dfd1b4d7-5544-40c0-af2a-f22db9888c91-kube-api-access-xnxmh\") pod \"node-resolver-rxdnb\" (UID: \"dfd1b4d7-5544-40c0-af2a-f22db9888c91\") " pod="openshift-dns/node-resolver-rxdnb" Jul 08 18:18:01.707629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706488 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/506c4790-d442-4464-8b83-06d2891a6df0-kubelet-config\") pod \"global-pull-secret-syncer-xw9f2\" (UID: \"506c4790-d442-4464-8b83-06d2891a6df0\") " pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:01.707629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706509 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-hostroot\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.707629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706523 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/33915cb2-f6ca-4540-9425-ff653a28c8a2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.707629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706531 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33915cb2-f6ca-4540-9425-ff653a28c8a2-system-cni-dir\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.707629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706444 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-etc-kubernetes\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.707629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706547 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-sysctl-d\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.707629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706570 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a4e7ee8d-935d-4f12-a691-c123e9a5fd27-agent-certs\") pod \"konnectivity-agent-8svs8\" (UID: \"a4e7ee8d-935d-4f12-a691-c123e9a5fd27\") " pod="kube-system/konnectivity-agent-8svs8" Jul 08 18:18:01.707629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706573 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/506c4790-d442-4464-8b83-06d2891a6df0-kubelet-config\") pod \"global-pull-secret-syncer-xw9f2\" (UID: \"506c4790-d442-4464-8b83-06d2891a6df0\") " pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:01.707629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706590 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a4e7ee8d-935d-4f12-a691-c123e9a5fd27-konnectivity-ca\") pod \"konnectivity-agent-8svs8\" (UID: \"a4e7ee8d-935d-4f12-a691-c123e9a5fd27\") " pod="kube-system/konnectivity-agent-8svs8" Jul 08 18:18:01.707629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706594 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33915cb2-f6ca-4540-9425-ff653a28c8a2-system-cni-dir\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.707629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706627 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-hostroot\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.707629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706632 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-sys-fs\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.707629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706629 2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 18:18:01.707629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706741 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkjrm\" (UniqueName: \"kubernetes.io/projected/f3fd4161-4fee-43ef-ad4a-e784880b494a-kube-api-access-bkjrm\") pod \"iptables-alerter-pt8xd\" (UID: \"f3fd4161-4fee-43ef-ad4a-e784880b494a\") " pod="openshift-network-operator/iptables-alerter-pt8xd" Jul 08 18:18:01.707629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706765 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-run-netns\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.708314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706799 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-cni-bin\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.708314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706824 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs\") pod \"network-metrics-daemon-ts8sd\" (UID: \"0d868453-b10b-4dd9-b843-91de3f1b48d3\") " pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:01.708314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706871 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-multus-conf-dir\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.708314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706893 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a05de7da-b5fb-4f8b-b52f-d9222471fae2-ovnkube-config\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.708314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706946 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-tmp\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.708314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706948 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-multus-conf-dir\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.708314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706968 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-os-release\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.708314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.706996 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ff6bcc41-a870-4794-96f3-418b31e9a666-multus-daemon-config\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.708314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707002 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-sys-fs\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.708314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707025 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpbgx\" (UniqueName: \"kubernetes.io/projected/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-kube-api-access-lpbgx\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.708314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707053 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-run-systemd\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.708314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707084 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a4e7ee8d-935d-4f12-a691-c123e9a5fd27-konnectivity-ca\") pod \"konnectivity-agent-8svs8\" (UID: \"a4e7ee8d-935d-4f12-a691-c123e9a5fd27\") " pod="kube-system/konnectivity-agent-8svs8" Jul 08 18:18:01.708314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707128 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a05de7da-b5fb-4f8b-b52f-d9222471fae2-ovnkube-script-lib\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.708314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707187 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/33915cb2-f6ca-4540-9425-ff653a28c8a2-os-release\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.708314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707213 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-modprobe-d\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.708314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707239 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-run\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.708314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707267 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-run-ovn\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.709132 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707287 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ec2217c0-db7c-4660-a6f3-990b90cbf03d-serviceca\") pod \"node-ca-smjlq\" (UID: \"ec2217c0-db7c-4660-a6f3-990b90cbf03d\") " pod="openshift-image-registry/node-ca-smjlq" Jul 08 18:18:01.709132 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707347 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret\") pod \"global-pull-secret-syncer-xw9f2\" (UID: \"506c4790-d442-4464-8b83-06d2891a6df0\") " pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:01.709132 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707363 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a05de7da-b5fb-4f8b-b52f-d9222471fae2-ovn-node-metrics-cert\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.709132 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707379 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfd1b4d7-5544-40c0-af2a-f22db9888c91-tmp-dir\") pod \"node-resolver-rxdnb\" (UID: \"dfd1b4d7-5544-40c0-af2a-f22db9888c91\") " pod="openshift-dns/node-resolver-rxdnb" Jul 08 18:18:01.709132 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707398 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec2217c0-db7c-4660-a6f3-990b90cbf03d-host\") pod \"node-ca-smjlq\" (UID: \"ec2217c0-db7c-4660-a6f3-990b90cbf03d\") " pod="openshift-image-registry/node-ca-smjlq" Jul 08 18:18:01.709132 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707419 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-etc-selinux\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.709132 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707437 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-systemd-units\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.709132 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707456 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-slash\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.709132 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707471 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.709132 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707490 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/506c4790-d442-4464-8b83-06d2891a6df0-dbus\") pod \"global-pull-secret-syncer-xw9f2\" (UID: \"506c4790-d442-4464-8b83-06d2891a6df0\") " pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:01.709132 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707494 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ff6bcc41-a870-4794-96f3-418b31e9a666-multus-daemon-config\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.709132 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707506 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-run-multus-certs\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.709132 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707523 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt9bc\" (UniqueName: \"kubernetes.io/projected/ff6bcc41-a870-4794-96f3-418b31e9a666-kube-api-access-xt9bc\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.709132 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707614 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-os-release\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.709132 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707656 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tx9d\" (UniqueName: \"kubernetes.io/projected/a05de7da-b5fb-4f8b-b52f-d9222471fae2-kube-api-access-2tx9d\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.709132 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707674 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/33915cb2-f6ca-4540-9425-ff653a28c8a2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.709132 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707689 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-var-lib-cni-bin\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.709925 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707749 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-var-lib-cni-bin\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.709925 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707760 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/33915cb2-f6ca-4540-9425-ff653a28c8a2-os-release\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.709925 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707827 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-modprobe-d\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.709925 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707822 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-run\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.709925 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707946 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ff6bcc41-a870-4794-96f3-418b31e9a666-host-run-multus-certs\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.709925 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.707966 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/506c4790-d442-4464-8b83-06d2891a6df0-dbus\") pod \"global-pull-secret-syncer-xw9f2\" (UID: \"506c4790-d442-4464-8b83-06d2891a6df0\") " pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:01.709925 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:01.707976 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:01.709925 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:01.708083 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:01.709925 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:01.708338 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret podName:506c4790-d442-4464-8b83-06d2891a6df0 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:02.208317672 +0000 UTC m=+2.959627140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret") pod "global-pull-secret-syncer-xw9f2" (UID: "506c4790-d442-4464-8b83-06d2891a6df0") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:01.709925 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:01.708359 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs podName:0d868453-b10b-4dd9-b843-91de3f1b48d3 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:02.208348789 +0000 UTC m=+2.959658247 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs") pod "network-metrics-daemon-ts8sd" (UID: "0d868453-b10b-4dd9-b843-91de3f1b48d3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:01.709925 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.708386 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aaf14a01-c937-4cd9-aa30-e1de026e5680-etc-selinux\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.709925 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.708511 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/33915cb2-f6ca-4540-9425-ff653a28c8a2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.710372 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.710119 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-etc-tuned\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.710372 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.710181 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-tmp\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.710372 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.710359 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a4e7ee8d-935d-4f12-a691-c123e9a5fd27-agent-certs\") pod \"konnectivity-agent-8svs8\" (UID: \"a4e7ee8d-935d-4f12-a691-c123e9a5fd27\") " pod="kube-system/konnectivity-agent-8svs8" Jul 08 18:18:01.715660 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.715624 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdg8p\" (UniqueName: \"kubernetes.io/projected/33915cb2-f6ca-4540-9425-ff653a28c8a2-kube-api-access-bdg8p\") pod \"multus-additional-cni-plugins-tmdn8\" (UID: \"33915cb2-f6ca-4540-9425-ff653a28c8a2\") " pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.716769 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:01.716747 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:18:01.716903 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:01.716775 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:18:01.716903 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:01.716790 2574 projected.go:196] Error preparing data for projected volume kube-api-access-6g5wf for pod openshift-network-diagnostics/network-check-target-fq9lp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:01.716903 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.716831 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt9bc\" (UniqueName: \"kubernetes.io/projected/ff6bcc41-a870-4794-96f3-418b31e9a666-kube-api-access-xt9bc\") pod \"multus-dkrkt\" (UID: \"ff6bcc41-a870-4794-96f3-418b31e9a666\") " pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.717066 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.716994 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shjkn\" (UniqueName: \"kubernetes.io/projected/aaf14a01-c937-4cd9-aa30-e1de026e5680-kube-api-access-shjkn\") pod \"aws-ebs-csi-driver-node-z6nl4\" (UID: \"aaf14a01-c937-4cd9-aa30-e1de026e5680\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.717296 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:01.717275 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf podName:3a8d7a1d-49b0-41a0-a297-2898090ea58f nodeName:}" failed. No retries permitted until 2026-07-08 18:18:02.217252138 +0000 UTC m=+2.968561605 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6g5wf" (UniqueName: "kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf") pod "network-check-target-fq9lp" (UID: "3a8d7a1d-49b0-41a0-a297-2898090ea58f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:01.718790 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.718746 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkjrm\" (UniqueName: \"kubernetes.io/projected/f3fd4161-4fee-43ef-ad4a-e784880b494a-kube-api-access-bkjrm\") pod \"iptables-alerter-pt8xd\" (UID: \"f3fd4161-4fee-43ef-ad4a-e784880b494a\") " pod="openshift-network-operator/iptables-alerter-pt8xd" Jul 08 18:18:01.719297 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.719278 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwn8k\" (UniqueName: \"kubernetes.io/projected/0d868453-b10b-4dd9-b843-91de3f1b48d3-kube-api-access-qwn8k\") pod \"network-metrics-daemon-ts8sd\" (UID: \"0d868453-b10b-4dd9-b843-91de3f1b48d3\") " pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:01.719558 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.719531 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpbgx\" (UniqueName: \"kubernetes.io/projected/b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4-kube-api-access-lpbgx\") pod \"tuned-hst6g\" (UID: \"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4\") " pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.768814 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.768756 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal" event={"ID":"c38e96b9a88058f800f8db7067621905","Type":"ContainerStarted","Data":"b2a9cf2dd41bb335e9f402eecc4d00e6abfd75aba51d36fc05f534ca508434b2"} Jul 08 18:18:01.769978 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.769951 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-97.ec2.internal" event={"ID":"069618fd9935d30ef59a0acde000b9cb","Type":"ContainerStarted","Data":"4e405ea9b50fc760c21e4b6d91e52510b43fb57de90b5e2770b750c9df9d5918"} Jul 08 18:18:01.808939 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.808899 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tx9d\" (UniqueName: \"kubernetes.io/projected/a05de7da-b5fb-4f8b-b52f-d9222471fae2-kube-api-access-2tx9d\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.808939 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.808945 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-kubelet\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.808968 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-var-lib-openvswitch\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.808994 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-etc-openvswitch\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809014 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-run-openvswitch\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809035 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-cni-netd\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809045 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-kubelet\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809059 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a05de7da-b5fb-4f8b-b52f-d9222471fae2-env-overrides\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809045 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-var-lib-openvswitch\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809080 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-run-openvswitch\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809089 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-etc-openvswitch\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809084 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dfd1b4d7-5544-40c0-af2a-f22db9888c91-hosts-file\") pod \"node-resolver-rxdnb\" (UID: \"dfd1b4d7-5544-40c0-af2a-f22db9888c91\") " pod="openshift-dns/node-resolver-rxdnb" Jul 08 18:18:01.809172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809122 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-cni-netd\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809154 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-node-log\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809185 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-run-ovn-kubernetes\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809200 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dfd1b4d7-5544-40c0-af2a-f22db9888c91-hosts-file\") pod \"node-resolver-rxdnb\" (UID: \"dfd1b4d7-5544-40c0-af2a-f22db9888c91\") " pod="openshift-dns/node-resolver-rxdnb" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809211 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjpxm\" (UniqueName: \"kubernetes.io/projected/ec2217c0-db7c-4660-a6f3-990b90cbf03d-kube-api-access-rjpxm\") pod \"node-ca-smjlq\" (UID: \"ec2217c0-db7c-4660-a6f3-990b90cbf03d\") " pod="openshift-image-registry/node-ca-smjlq" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809246 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-node-log\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809251 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-run-ovn-kubernetes\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809293 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-log-socket\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809322 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnxmh\" (UniqueName: \"kubernetes.io/projected/dfd1b4d7-5544-40c0-af2a-f22db9888c91-kube-api-access-xnxmh\") pod \"node-resolver-rxdnb\" (UID: \"dfd1b4d7-5544-40c0-af2a-f22db9888c91\") " pod="openshift-dns/node-resolver-rxdnb" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809354 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-run-netns\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809377 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-cni-bin\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809355 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-log-socket\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809409 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a05de7da-b5fb-4f8b-b52f-d9222471fae2-ovnkube-config\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809451 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-run-systemd\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809467 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a05de7da-b5fb-4f8b-b52f-d9222471fae2-ovnkube-script-lib\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809493 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-run-ovn\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809516 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ec2217c0-db7c-4660-a6f3-990b90cbf03d-serviceca\") pod \"node-ca-smjlq\" (UID: \"ec2217c0-db7c-4660-a6f3-990b90cbf03d\") " pod="openshift-image-registry/node-ca-smjlq" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809547 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a05de7da-b5fb-4f8b-b52f-d9222471fae2-ovn-node-metrics-cert\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809573 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfd1b4d7-5544-40c0-af2a-f22db9888c91-tmp-dir\") pod \"node-resolver-rxdnb\" (UID: \"dfd1b4d7-5544-40c0-af2a-f22db9888c91\") " pod="openshift-dns/node-resolver-rxdnb" Jul 08 18:18:01.809681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809580 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a05de7da-b5fb-4f8b-b52f-d9222471fae2-env-overrides\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.810298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809584 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-cni-bin\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.810298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809592 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec2217c0-db7c-4660-a6f3-990b90cbf03d-host\") pod \"node-ca-smjlq\" (UID: \"ec2217c0-db7c-4660-a6f3-990b90cbf03d\") " pod="openshift-image-registry/node-ca-smjlq" Jul 08 18:18:01.810298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809622 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-systemd-units\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.810298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809627 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-run-ovn\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.810298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809636 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-slash\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.810298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809652 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-run-netns\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.810298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809660 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.810298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809679 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-run-systemd\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.810298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809726 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.810298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809933 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a05de7da-b5fb-4f8b-b52f-d9222471fae2-ovnkube-config\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.810298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.809971 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-systemd-units\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.810298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.810000 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfd1b4d7-5544-40c0-af2a-f22db9888c91-tmp-dir\") pod \"node-resolver-rxdnb\" (UID: \"dfd1b4d7-5544-40c0-af2a-f22db9888c91\") " pod="openshift-dns/node-resolver-rxdnb" Jul 08 18:18:01.810298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.810012 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec2217c0-db7c-4660-a6f3-990b90cbf03d-host\") pod \"node-ca-smjlq\" (UID: \"ec2217c0-db7c-4660-a6f3-990b90cbf03d\") " pod="openshift-image-registry/node-ca-smjlq" Jul 08 18:18:01.810298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.810042 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a05de7da-b5fb-4f8b-b52f-d9222471fae2-host-slash\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.810298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.810064 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a05de7da-b5fb-4f8b-b52f-d9222471fae2-ovnkube-script-lib\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.810298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.810271 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ec2217c0-db7c-4660-a6f3-990b90cbf03d-serviceca\") pod \"node-ca-smjlq\" (UID: \"ec2217c0-db7c-4660-a6f3-990b90cbf03d\") " pod="openshift-image-registry/node-ca-smjlq" Jul 08 18:18:01.812062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.812040 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a05de7da-b5fb-4f8b-b52f-d9222471fae2-ovn-node-metrics-cert\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.820867 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.820826 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjpxm\" (UniqueName: \"kubernetes.io/projected/ec2217c0-db7c-4660-a6f3-990b90cbf03d-kube-api-access-rjpxm\") pod \"node-ca-smjlq\" (UID: \"ec2217c0-db7c-4660-a6f3-990b90cbf03d\") " pod="openshift-image-registry/node-ca-smjlq" Jul 08 18:18:01.822311 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.822290 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnxmh\" (UniqueName: \"kubernetes.io/projected/dfd1b4d7-5544-40c0-af2a-f22db9888c91-kube-api-access-xnxmh\") pod \"node-resolver-rxdnb\" (UID: \"dfd1b4d7-5544-40c0-af2a-f22db9888c91\") " pod="openshift-dns/node-resolver-rxdnb" Jul 08 18:18:01.823122 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.823103 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tx9d\" (UniqueName: \"kubernetes.io/projected/a05de7da-b5fb-4f8b-b52f-d9222471fae2-kube-api-access-2tx9d\") pod \"ovnkube-node-x56pf\" (UID: \"a05de7da-b5fb-4f8b-b52f-d9222471fae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:01.896591 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.896498 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pt8xd" Jul 08 18:18:01.903021 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.902997 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tmdn8" Jul 08 18:18:01.910690 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.910665 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dkrkt" Jul 08 18:18:01.915309 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.915290 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hst6g" Jul 08 18:18:01.921705 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.921681 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8svs8" Jul 08 18:18:01.927268 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.927246 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" Jul 08 18:18:01.932681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.932658 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rxdnb" Jul 08 18:18:01.938254 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.938231 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-smjlq" Jul 08 18:18:01.944912 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:01.944895 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:02.089221 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:02.089194 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:18:02.212949 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:02.212876 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs\") pod \"network-metrics-daemon-ts8sd\" (UID: \"0d868453-b10b-4dd9-b843-91de3f1b48d3\") " pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:02.212949 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:02.212911 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret\") pod \"global-pull-secret-syncer-xw9f2\" (UID: \"506c4790-d442-4464-8b83-06d2891a6df0\") " pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:02.213161 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:02.213008 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:02.213161 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:02.213033 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:02.213161 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:02.213057 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret podName:506c4790-d442-4464-8b83-06d2891a6df0 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:03.213044581 +0000 UTC m=+3.964354034 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret") pod "global-pull-secret-syncer-xw9f2" (UID: "506c4790-d442-4464-8b83-06d2891a6df0") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:02.213161 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:02.213089 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs podName:0d868453-b10b-4dd9-b843-91de3f1b48d3 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:03.213072963 +0000 UTC m=+3.964382420 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs") pod "network-metrics-daemon-ts8sd" (UID: "0d868453-b10b-4dd9-b843-91de3f1b48d3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:02.313468 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:02.313434 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g5wf\" (UniqueName: \"kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf\") pod \"network-check-target-fq9lp\" (UID: \"3a8d7a1d-49b0-41a0-a297-2898090ea58f\") " pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:02.313639 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:02.313616 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:18:02.313685 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:02.313646 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:18:02.313685 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:02.313656 2574 projected.go:196] Error preparing data for projected volume kube-api-access-6g5wf for pod openshift-network-diagnostics/network-check-target-fq9lp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:02.313751 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:02.313704 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf podName:3a8d7a1d-49b0-41a0-a297-2898090ea58f nodeName:}" failed. No retries permitted until 2026-07-08 18:18:03.313691451 +0000 UTC m=+4.065000904 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6g5wf" (UniqueName: "kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf") pod "network-check-target-fq9lp" (UID: "3a8d7a1d-49b0-41a0-a297-2898090ea58f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:02.410272 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:18:02.410241 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaf14a01_c937_4cd9_aa30_e1de026e5680.slice/crio-1ebb01638a948b2d66a6fc34c1cfc94678c975922600086f4fa3575dbc364908 WatchSource:0}: Error finding container 1ebb01638a948b2d66a6fc34c1cfc94678c975922600086f4fa3575dbc364908: Status 404 returned error can't find the container with id 1ebb01638a948b2d66a6fc34c1cfc94678c975922600086f4fa3575dbc364908 Jul 08 18:18:02.411054 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:18:02.410972 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33915cb2_f6ca_4540_9425_ff653a28c8a2.slice/crio-657597bc020398753ea52281d32680a8c5c27e67e48f48ceb1355d941177f702 WatchSource:0}: Error finding container 657597bc020398753ea52281d32680a8c5c27e67e48f48ceb1355d941177f702: Status 404 returned error can't find the container with id 657597bc020398753ea52281d32680a8c5c27e67e48f48ceb1355d941177f702 Jul 08 18:18:02.412515 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:18:02.412490 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff6bcc41_a870_4794_96f3_418b31e9a666.slice/crio-8cbb4e087a0b59ab826c3dcb3367c0249a7e8ec74957f38b566e83a225b3e3e2 WatchSource:0}: Error finding container 8cbb4e087a0b59ab826c3dcb3367c0249a7e8ec74957f38b566e83a225b3e3e2: Status 404 returned error can't find the container with id 8cbb4e087a0b59ab826c3dcb3367c0249a7e8ec74957f38b566e83a225b3e3e2 Jul 08 18:18:02.415635 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:18:02.415618 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e7ee8d_935d_4f12_a691_c123e9a5fd27.slice/crio-62aafc1f2a9b9f931c4fd9a9fcd942ec83b6ae142b148f4d7c63d2076577d3df WatchSource:0}: Error finding container 62aafc1f2a9b9f931c4fd9a9fcd942ec83b6ae142b148f4d7c63d2076577d3df: Status 404 returned error can't find the container with id 62aafc1f2a9b9f931c4fd9a9fcd942ec83b6ae142b148f4d7c63d2076577d3df Jul 08 18:18:02.416962 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:18:02.416933 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda05de7da_b5fb_4f8b_b52f_d9222471fae2.slice/crio-9c76403eeb5766bc9f540d2974182951669c6e266fe0e739abb1f7414005b7fe WatchSource:0}: Error finding container 9c76403eeb5766bc9f540d2974182951669c6e266fe0e739abb1f7414005b7fe: Status 404 returned error can't find the container with id 9c76403eeb5766bc9f540d2974182951669c6e266fe0e739abb1f7414005b7fe Jul 08 18:18:02.417839 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:18:02.417818 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfd1b4d7_5544_40c0_af2a_f22db9888c91.slice/crio-79e8b3cd6dd3b8840a25f16a42beea47f365d4b5e773b2cd15c8537b60ac87fc WatchSource:0}: Error finding container 79e8b3cd6dd3b8840a25f16a42beea47f365d4b5e773b2cd15c8537b60ac87fc: Status 404 returned error can't find the container with id 79e8b3cd6dd3b8840a25f16a42beea47f365d4b5e773b2cd15c8537b60ac87fc Jul 08 18:18:02.418543 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:18:02.418523 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3fd4161_4fee_43ef_ad4a_e784880b494a.slice/crio-a850f8b85c40ebf748fbd5e843d79ca5085edce04ef72f7a43ccc8f96eea39a1 WatchSource:0}: Error finding container a850f8b85c40ebf748fbd5e843d79ca5085edce04ef72f7a43ccc8f96eea39a1: Status 404 returned error can't find the container with id a850f8b85c40ebf748fbd5e843d79ca5085edce04ef72f7a43ccc8f96eea39a1 Jul 08 18:18:02.420931 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:18:02.420910 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb83b2ce0_f102_420f_bdf1_cd6cbb30bdf4.slice/crio-de07d369780060eed95c26d859ce6f7764550b4b5baa5c089865a996ae726fd6 WatchSource:0}: Error finding container de07d369780060eed95c26d859ce6f7764550b4b5baa5c089865a996ae726fd6: Status 404 returned error can't find the container with id de07d369780060eed95c26d859ce6f7764550b4b5baa5c089865a996ae726fd6 Jul 08 18:18:02.421141 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:18:02.421124 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec2217c0_db7c_4660_a6f3_990b90cbf03d.slice/crio-f404ea6e6a83d4e67939831bafe3329b17ebf38b8ff23667ab3f580e15b253d6 WatchSource:0}: Error finding container f404ea6e6a83d4e67939831bafe3329b17ebf38b8ff23667ab3f580e15b253d6: Status 404 returned error can't find the container with id f404ea6e6a83d4e67939831bafe3329b17ebf38b8ff23667ab3f580e15b253d6 Jul 08 18:18:02.641665 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:02.641478 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 18:13:01 +0000 UTC" deadline="2028-04-03 09:20:44.511009616 +0000 UTC" Jul 08 18:18:02.642028 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:02.641669 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15231h2m41.869348881s" Jul 08 18:18:02.773536 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:02.773426 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-97.ec2.internal" event={"ID":"069618fd9935d30ef59a0acde000b9cb","Type":"ContainerStarted","Data":"af422c71d12ad3a8f9f8087c8c8a77e74d001cdaaacb2fa03564c3863b7e4711"} Jul 08 18:18:02.774601 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:02.774567 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hst6g" event={"ID":"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4","Type":"ContainerStarted","Data":"de07d369780060eed95c26d859ce6f7764550b4b5baa5c089865a996ae726fd6"} Jul 08 18:18:02.775791 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:02.775765 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" event={"ID":"a05de7da-b5fb-4f8b-b52f-d9222471fae2","Type":"ContainerStarted","Data":"9c76403eeb5766bc9f540d2974182951669c6e266fe0e739abb1f7414005b7fe"} Jul 08 18:18:02.777061 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:02.777036 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pt8xd" event={"ID":"f3fd4161-4fee-43ef-ad4a-e784880b494a","Type":"ContainerStarted","Data":"a850f8b85c40ebf748fbd5e843d79ca5085edce04ef72f7a43ccc8f96eea39a1"} Jul 08 18:18:02.778331 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:02.778310 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-smjlq" event={"ID":"ec2217c0-db7c-4660-a6f3-990b90cbf03d","Type":"ContainerStarted","Data":"f404ea6e6a83d4e67939831bafe3329b17ebf38b8ff23667ab3f580e15b253d6"} Jul 08 18:18:02.780286 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:02.780262 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmdn8" event={"ID":"33915cb2-f6ca-4540-9425-ff653a28c8a2","Type":"ContainerStarted","Data":"657597bc020398753ea52281d32680a8c5c27e67e48f48ceb1355d941177f702"} Jul 08 18:18:02.781462 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:02.781438 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dkrkt" event={"ID":"ff6bcc41-a870-4794-96f3-418b31e9a666","Type":"ContainerStarted","Data":"8cbb4e087a0b59ab826c3dcb3367c0249a7e8ec74957f38b566e83a225b3e3e2"} Jul 08 18:18:02.782665 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:02.782632 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rxdnb" event={"ID":"dfd1b4d7-5544-40c0-af2a-f22db9888c91","Type":"ContainerStarted","Data":"79e8b3cd6dd3b8840a25f16a42beea47f365d4b5e773b2cd15c8537b60ac87fc"} Jul 08 18:18:02.783807 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:02.783784 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8svs8" event={"ID":"a4e7ee8d-935d-4f12-a691-c123e9a5fd27","Type":"ContainerStarted","Data":"62aafc1f2a9b9f931c4fd9a9fcd942ec83b6ae142b148f4d7c63d2076577d3df"} Jul 08 18:18:02.784889 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:02.784870 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" event={"ID":"aaf14a01-c937-4cd9-aa30-e1de026e5680","Type":"ContainerStarted","Data":"1ebb01638a948b2d66a6fc34c1cfc94678c975922600086f4fa3575dbc364908"} Jul 08 18:18:03.219538 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:03.219453 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs\") pod \"network-metrics-daemon-ts8sd\" (UID: \"0d868453-b10b-4dd9-b843-91de3f1b48d3\") " pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:03.219538 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:03.219515 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret\") pod \"global-pull-secret-syncer-xw9f2\" (UID: \"506c4790-d442-4464-8b83-06d2891a6df0\") " pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:03.219756 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:03.219639 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:03.219756 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:03.219698 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret podName:506c4790-d442-4464-8b83-06d2891a6df0 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:05.219681267 +0000 UTC m=+5.970990722 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret") pod "global-pull-secret-syncer-xw9f2" (UID: "506c4790-d442-4464-8b83-06d2891a6df0") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:03.220121 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:03.220104 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:03.220203 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:03.220159 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs podName:0d868453-b10b-4dd9-b843-91de3f1b48d3 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:05.220141262 +0000 UTC m=+5.971450714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs") pod "network-metrics-daemon-ts8sd" (UID: "0d868453-b10b-4dd9-b843-91de3f1b48d3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:03.320582 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:03.320541 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g5wf\" (UniqueName: \"kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf\") pod \"network-check-target-fq9lp\" (UID: \"3a8d7a1d-49b0-41a0-a297-2898090ea58f\") " pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:03.321367 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:03.320912 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:18:03.321367 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:03.320937 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:18:03.321367 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:03.320950 2574 projected.go:196] Error preparing data for projected volume kube-api-access-6g5wf for pod openshift-network-diagnostics/network-check-target-fq9lp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:03.321367 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:03.321008 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf podName:3a8d7a1d-49b0-41a0-a297-2898090ea58f nodeName:}" failed. No retries permitted until 2026-07-08 18:18:05.320990525 +0000 UTC m=+6.072299982 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6g5wf" (UniqueName: "kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf") pod "network-check-target-fq9lp" (UID: "3a8d7a1d-49b0-41a0-a297-2898090ea58f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:03.642510 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:03.642358 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 18:13:01 +0000 UTC" deadline="2028-01-29 12:02:04.63393062 +0000 UTC" Jul 08 18:18:03.642510 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:03.642442 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13673h44m0.991492713s" Jul 08 18:18:03.764565 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:03.764535 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:03.764732 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:03.764535 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:03.764732 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:03.764666 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xw9f2" podUID="506c4790-d442-4464-8b83-06d2891a6df0" Jul 08 18:18:03.764843 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:03.764748 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:03.764843 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:03.764535 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:03.764961 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:03.764871 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:05.240210 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:05.239437 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs\") pod \"network-metrics-daemon-ts8sd\" (UID: \"0d868453-b10b-4dd9-b843-91de3f1b48d3\") " pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:05.240210 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:05.239491 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret\") pod \"global-pull-secret-syncer-xw9f2\" (UID: \"506c4790-d442-4464-8b83-06d2891a6df0\") " pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:05.240210 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:05.239652 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:05.240210 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:05.239716 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret podName:506c4790-d442-4464-8b83-06d2891a6df0 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:09.239698624 +0000 UTC m=+9.991008092 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret") pod "global-pull-secret-syncer-xw9f2" (UID: "506c4790-d442-4464-8b83-06d2891a6df0") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:05.240210 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:05.240116 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:05.240210 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:05.240165 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs podName:0d868453-b10b-4dd9-b843-91de3f1b48d3 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:09.240150017 +0000 UTC m=+9.991459475 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs") pod "network-metrics-daemon-ts8sd" (UID: "0d868453-b10b-4dd9-b843-91de3f1b48d3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:05.340831 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:05.340791 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g5wf\" (UniqueName: \"kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf\") pod \"network-check-target-fq9lp\" (UID: \"3a8d7a1d-49b0-41a0-a297-2898090ea58f\") " pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:05.341029 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:05.340984 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:18:05.341029 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:05.341003 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:18:05.341029 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:05.341016 2574 projected.go:196] Error preparing data for projected volume kube-api-access-6g5wf for pod openshift-network-diagnostics/network-check-target-fq9lp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:05.341198 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:05.341072 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf podName:3a8d7a1d-49b0-41a0-a297-2898090ea58f nodeName:}" failed. No retries permitted until 2026-07-08 18:18:09.341055473 +0000 UTC m=+10.092364930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6g5wf" (UniqueName: "kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf") pod "network-check-target-fq9lp" (UID: "3a8d7a1d-49b0-41a0-a297-2898090ea58f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:05.766282 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:05.766236 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:05.766454 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:05.766361 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:05.766999 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:05.766977 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:05.767130 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:05.767081 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xw9f2" podUID="506c4790-d442-4464-8b83-06d2891a6df0" Jul 08 18:18:05.767195 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:05.767175 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:05.767280 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:05.767258 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:07.767127 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:07.764777 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:07.767127 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:07.764814 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:07.767127 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:07.764777 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:07.767127 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:07.764934 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xw9f2" podUID="506c4790-d442-4464-8b83-06d2891a6df0" Jul 08 18:18:07.767127 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:07.765370 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:07.767127 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:07.765483 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:09.277615 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:09.277571 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs\") pod \"network-metrics-daemon-ts8sd\" (UID: \"0d868453-b10b-4dd9-b843-91de3f1b48d3\") " pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:09.278098 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:09.277631 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret\") pod \"global-pull-secret-syncer-xw9f2\" (UID: \"506c4790-d442-4464-8b83-06d2891a6df0\") " pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:09.278098 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:09.277791 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:09.278098 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:09.277841 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:09.278098 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:09.277890 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs podName:0d868453-b10b-4dd9-b843-91de3f1b48d3 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:17.277869931 +0000 UTC m=+18.029179388 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs") pod "network-metrics-daemon-ts8sd" (UID: "0d868453-b10b-4dd9-b843-91de3f1b48d3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:09.278098 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:09.277921 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret podName:506c4790-d442-4464-8b83-06d2891a6df0 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:17.27790419 +0000 UTC m=+18.029213648 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret") pod "global-pull-secret-syncer-xw9f2" (UID: "506c4790-d442-4464-8b83-06d2891a6df0") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:09.378441 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:09.378321 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g5wf\" (UniqueName: \"kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf\") pod \"network-check-target-fq9lp\" (UID: \"3a8d7a1d-49b0-41a0-a297-2898090ea58f\") " pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:09.378618 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:09.378495 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:18:09.378618 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:09.378518 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:18:09.378618 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:09.378532 2574 projected.go:196] Error preparing data for projected volume kube-api-access-6g5wf for pod openshift-network-diagnostics/network-check-target-fq9lp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:09.378618 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:09.378598 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf podName:3a8d7a1d-49b0-41a0-a297-2898090ea58f nodeName:}" failed. No retries permitted until 2026-07-08 18:18:17.37857779 +0000 UTC m=+18.129887254 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6g5wf" (UniqueName: "kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf") pod "network-check-target-fq9lp" (UID: "3a8d7a1d-49b0-41a0-a297-2898090ea58f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:09.766385 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:09.765638 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:09.766385 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:09.765738 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xw9f2" podUID="506c4790-d442-4464-8b83-06d2891a6df0" Jul 08 18:18:09.766385 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:09.766092 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:09.766385 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:09.766172 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:09.766385 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:09.766221 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:09.766385 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:09.766300 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:11.766001 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:11.765967 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:11.766439 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:11.765967 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:11.766439 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:11.766093 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:11.766439 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:11.766189 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:11.766439 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:11.766194 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xw9f2" podUID="506c4790-d442-4464-8b83-06d2891a6df0" Jul 08 18:18:11.766439 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:11.766263 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:13.765343 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:13.765261 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:13.765708 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:13.765263 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:13.765708 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:13.765467 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xw9f2" podUID="506c4790-d442-4464-8b83-06d2891a6df0" Jul 08 18:18:13.765708 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:13.765372 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:13.765708 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:13.765501 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:13.765708 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:13.765576 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:15.764573 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:15.764542 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:15.765053 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:15.764582 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:15.765053 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:15.764550 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:15.765053 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:15.764693 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:15.765053 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:15.764770 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xw9f2" podUID="506c4790-d442-4464-8b83-06d2891a6df0" Jul 08 18:18:15.765053 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:15.764840 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:17.338932 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:17.338889 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs\") pod \"network-metrics-daemon-ts8sd\" (UID: \"0d868453-b10b-4dd9-b843-91de3f1b48d3\") " pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:17.339459 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:17.338947 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret\") pod \"global-pull-secret-syncer-xw9f2\" (UID: \"506c4790-d442-4464-8b83-06d2891a6df0\") " pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:17.339459 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:17.339052 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:17.339459 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:17.339090 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:17.339459 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:17.339134 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs podName:0d868453-b10b-4dd9-b843-91de3f1b48d3 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:33.339113127 +0000 UTC m=+34.090422582 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs") pod "network-metrics-daemon-ts8sd" (UID: "0d868453-b10b-4dd9-b843-91de3f1b48d3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:17.339459 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:17.339156 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret podName:506c4790-d442-4464-8b83-06d2891a6df0 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:33.339145114 +0000 UTC m=+34.090454570 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret") pod "global-pull-secret-syncer-xw9f2" (UID: "506c4790-d442-4464-8b83-06d2891a6df0") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:17.440289 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:17.440259 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g5wf\" (UniqueName: \"kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf\") pod \"network-check-target-fq9lp\" (UID: \"3a8d7a1d-49b0-41a0-a297-2898090ea58f\") " pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:17.440465 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:17.440412 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:18:17.440465 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:17.440431 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:18:17.440465 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:17.440442 2574 projected.go:196] Error preparing data for projected volume kube-api-access-6g5wf for pod openshift-network-diagnostics/network-check-target-fq9lp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:17.440627 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:17.440505 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf podName:3a8d7a1d-49b0-41a0-a297-2898090ea58f nodeName:}" failed. No retries permitted until 2026-07-08 18:18:33.440487539 +0000 UTC m=+34.191797004 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6g5wf" (UniqueName: "kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf") pod "network-check-target-fq9lp" (UID: "3a8d7a1d-49b0-41a0-a297-2898090ea58f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:17.767000 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:17.766966 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:17.767170 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:17.766966 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:17.767170 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:17.767074 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:17.767170 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:17.766966 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:17.767294 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:17.767145 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xw9f2" podUID="506c4790-d442-4464-8b83-06d2891a6df0" Jul 08 18:18:17.767294 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:17.767250 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:19.766109 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:19.766072 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:19.766580 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:19.766186 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xw9f2" podUID="506c4790-d442-4464-8b83-06d2891a6df0" Jul 08 18:18:19.766580 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:19.766202 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:19.766580 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:19.766218 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:19.766580 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:19.766326 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:19.766580 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:19.766418 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:21.765453 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:21.765419 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:21.765899 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:21.765419 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:21.765899 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:21.765533 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xw9f2" podUID="506c4790-d442-4464-8b83-06d2891a6df0" Jul 08 18:18:21.765899 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:21.765637 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:21.765899 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:21.765428 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:21.765899 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:21.765703 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:22.830568 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.830186 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" event={"ID":"aaf14a01-c937-4cd9-aa30-e1de026e5680","Type":"ContainerStarted","Data":"7fa6543ee1c43aa4fac8b62821be73a2785e9f57322eff3d26c2176f78cf954e"} Jul 08 18:18:22.831716 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.831687 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hst6g" event={"ID":"b83b2ce0-f102-420f-bdf1-cd6cbb30bdf4","Type":"ContainerStarted","Data":"65d20478452b1472d18eb35fa3db8c199b83f6a450d5c5643724614405c96e87"} Jul 08 18:18:22.834619 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.834598 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" event={"ID":"a05de7da-b5fb-4f8b-b52f-d9222471fae2","Type":"ContainerStarted","Data":"05c074bfa2f51752118c4ce321488f8ab1496dcf0c4d1ddb96f8fd836138d609"} Jul 08 18:18:22.834735 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.834666 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" event={"ID":"a05de7da-b5fb-4f8b-b52f-d9222471fae2","Type":"ContainerStarted","Data":"ebed9dcd34e307f9780e11340034a7b3c970059e27129f51f5994a23a5b95e1e"} Jul 08 18:18:22.834735 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.834715 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" event={"ID":"a05de7da-b5fb-4f8b-b52f-d9222471fae2","Type":"ContainerStarted","Data":"aa37c29e3d90b66cf0a140d1f3f025219a8653f631af94ed208bd716e89284d2"} Jul 08 18:18:22.834869 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.834755 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" event={"ID":"a05de7da-b5fb-4f8b-b52f-d9222471fae2","Type":"ContainerStarted","Data":"1cb8e8840cd9e088167516cc6062b96d9060d234b0125faec0399aa556ed92f2"} Jul 08 18:18:22.834869 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.834794 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" event={"ID":"a05de7da-b5fb-4f8b-b52f-d9222471fae2","Type":"ContainerStarted","Data":"d6da8fa1a55ac2172ced141e334b0fd5ce45773dd4ddda98273df8788ab3ef74"} Jul 08 18:18:22.834869 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.834843 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" event={"ID":"a05de7da-b5fb-4f8b-b52f-d9222471fae2","Type":"ContainerStarted","Data":"3417075f415f902e738c5c83c1f5fd3b5ddf6b9eec7fef0f67f7fc963e178f55"} Jul 08 18:18:22.835878 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.835833 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-smjlq" event={"ID":"ec2217c0-db7c-4660-a6f3-990b90cbf03d","Type":"ContainerStarted","Data":"8d0dd6db7e5e029cf5438d052d83136fa718eae2f1bf8bf6cb5e3add0967ed94"} Jul 08 18:18:22.837112 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.837091 2574 generic.go:356] "Generic (PLEG): container finished" podID="33915cb2-f6ca-4540-9425-ff653a28c8a2" containerID="6f18cdbd9386a5cf5c4f96f7142683da521e8bcbccd8b33b1021b70ff8e084b7" exitCode=0 Jul 08 18:18:22.837195 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.837153 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmdn8" event={"ID":"33915cb2-f6ca-4540-9425-ff653a28c8a2","Type":"ContainerDied","Data":"6f18cdbd9386a5cf5c4f96f7142683da521e8bcbccd8b33b1021b70ff8e084b7"} Jul 08 18:18:22.838566 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.838530 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dkrkt" event={"ID":"ff6bcc41-a870-4794-96f3-418b31e9a666","Type":"ContainerStarted","Data":"45c3d1c56a04f24dd72b9655edfef072d7141d1955cff3becf628fab3d0b1f90"} Jul 08 18:18:22.840076 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.840053 2574 generic.go:356] "Generic (PLEG): container finished" podID="c38e96b9a88058f800f8db7067621905" containerID="c6b278fb22052ed9600cdea367e6a91375b131db10fb712fcfd5b947f1dcbcaa" exitCode=0 Jul 08 18:18:22.840168 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.840109 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal" event={"ID":"c38e96b9a88058f800f8db7067621905","Type":"ContainerDied","Data":"c6b278fb22052ed9600cdea367e6a91375b131db10fb712fcfd5b947f1dcbcaa"} Jul 08 18:18:22.841564 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.841541 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rxdnb" event={"ID":"dfd1b4d7-5544-40c0-af2a-f22db9888c91","Type":"ContainerStarted","Data":"031001ff21be6269f83986befa74e5a861016baa22db9e630b12df97f6b67162"} Jul 08 18:18:22.842967 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.842946 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8svs8" event={"ID":"a4e7ee8d-935d-4f12-a691-c123e9a5fd27","Type":"ContainerStarted","Data":"f5acbcbe3e16fcf66c70dffedc4dbfdead48c4dfdc5897afdd2aca2e232677f7"} Jul 08 18:18:22.850585 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.850549 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-97.ec2.internal" podStartSLOduration=22.850537373 podStartE2EDuration="22.850537373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:18:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:02.787627718 +0000 UTC m=+3.538937192" watchObservedRunningTime="2026-07-08 18:18:22.850537373 +0000 UTC m=+23.601846846" Jul 08 18:18:22.850685 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.850609 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hst6g" podStartSLOduration=4.384188163 podStartE2EDuration="23.850604458s" totalImagesPullingTime="19.466416295s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:22.850126126 +0000 UTC m=+23.601435600" watchObservedRunningTime="2026-07-08 18:18:22.850604458 +0000 UTC m=+23.601914028" Jul 08 18:18:22.892008 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.891959 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8svs8" podStartSLOduration=4.420746235 podStartE2EDuration="23.891939451s" totalImagesPullingTime="19.471193216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:22.866879832 +0000 UTC m=+23.618189307" watchObservedRunningTime="2026-07-08 18:18:22.891939451 +0000 UTC m=+23.643248926" Jul 08 18:18:22.905333 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.905291 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rxdnb" podStartSLOduration=4.436414026 podStartE2EDuration="23.905278035s" totalImagesPullingTime="19.468864009s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:22.904978175 +0000 UTC m=+23.656287672" watchObservedRunningTime="2026-07-08 18:18:22.905278035 +0000 UTC m=+23.656587509" Jul 08 18:18:22.919658 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.919621 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-smjlq" podStartSLOduration=4.511820625 podStartE2EDuration="23.919607758s" totalImagesPullingTime="19.407787133s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:22.919529491 +0000 UTC m=+23.670838983" watchObservedRunningTime="2026-07-08 18:18:22.919607758 +0000 UTC m=+23.670917213" Jul 08 18:18:22.956784 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:22.956743 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dkrkt" podStartSLOduration=4.425263826 podStartE2EDuration="23.956729242s" totalImagesPullingTime="19.531465416s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:22.956596228 +0000 UTC m=+23.707905701" watchObservedRunningTime="2026-07-08 18:18:22.956729242 +0000 UTC m=+23.708038715" Jul 08 18:18:23.103313 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:23.103276 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:23.182273 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:23.182238 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:23.282499 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:23.282465 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:23.765505 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:23.765463 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:23.765685 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:23.765602 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:23.765685 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:23.765485 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:23.765802 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:23.765692 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xw9f2" podUID="506c4790-d442-4464-8b83-06d2891a6df0" Jul 08 18:18:23.765802 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:23.765462 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:23.765802 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:23.765770 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:23.846324 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:23.846290 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pt8xd" event={"ID":"f3fd4161-4fee-43ef-ad4a-e784880b494a","Type":"ContainerStarted","Data":"d2739f62d17eedc92ccadb789709370b660574b3fe950e55279e4048dd859430"} Jul 08 18:18:23.848564 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:23.848288 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal" event={"ID":"c38e96b9a88058f800f8db7067621905","Type":"ContainerStarted","Data":"ad5712e64c1ea11e4846d67fc83fa1157999c4fdb325e6f33ba35160d4f5f2fb"} Jul 08 18:18:23.876051 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:23.876008 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-97.ec2.internal" podStartSLOduration=23.875995636 podStartE2EDuration="23.875995636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:18:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:23.875971218 +0000 UTC m=+24.627280692" watchObservedRunningTime="2026-07-08 18:18:23.875995636 +0000 UTC m=+24.627305109" Jul 08 18:18:23.876168 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:23.876071 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pt8xd" podStartSLOduration=5.404343377 podStartE2EDuration="24.876066728s" totalImagesPullingTime="19.471723351s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:23.861083078 +0000 UTC m=+24.612392551" watchObservedRunningTime="2026-07-08 18:18:23.876066728 +0000 UTC m=+24.627376202" Jul 08 18:18:24.003254 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:24.003229 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:24.040950 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:24.040925 2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 18:18:24.103583 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:24.103545 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:24.304141 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:24.304075 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:24.404506 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:24.404469 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:24.477542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:24.477506 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:24.567323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:24.567234 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8svs8" Jul 08 18:18:24.567988 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:24.567969 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8svs8" Jul 08 18:18:24.660796 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:24.660681 2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T18:18:24.040943037Z","UUID":"122d327c-1801-4098-a193-6dbc80d92a81","Handler":null,"Name":"","Endpoint":""} Jul 08 18:18:24.662387 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:24.662364 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 18:18:24.662580 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:24.662410 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 18:18:24.853421 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:24.853346 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" event={"ID":"aaf14a01-c937-4cd9-aa30-e1de026e5680","Type":"ContainerStarted","Data":"9aa784e2ce4211fbff77727a5a3b71cdb8396caa19e9355e1c511513abbf83ea"} Jul 08 18:18:24.857885 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:24.857837 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" event={"ID":"a05de7da-b5fb-4f8b-b52f-d9222471fae2","Type":"ContainerStarted","Data":"92b2920ae9378a24a2542e2759ef4b3f9b2984cc63fe5357c560c376eb7d1c1d"} Jul 08 18:18:24.858178 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:24.858114 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8svs8" Jul 08 18:18:24.858893 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:24.858874 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8svs8" Jul 08 18:18:24.921824 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:24.921785 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:25.764833 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:25.764758 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:25.764833 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:25.764790 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:25.765065 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:25.764758 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:25.765065 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:25.764914 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:25.765065 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:25.764994 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:25.765065 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:25.765059 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xw9f2" podUID="506c4790-d442-4464-8b83-06d2891a6df0" Jul 08 18:18:25.861766 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:25.861730 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" event={"ID":"aaf14a01-c937-4cd9-aa30-e1de026e5680","Type":"ContainerStarted","Data":"0ef1975378f3e68a6316b715f7cc60c87cb85ff0ff08e2f01a6263edf30e575a"} Jul 08 18:18:25.879905 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:25.879828 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6nl4" podStartSLOduration=4.927527801 podStartE2EDuration="26.879810448s" totalImagesPullingTime="21.952282647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:17:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:25.879544383 +0000 UTC m=+26.630853848" watchObservedRunningTime="2026-07-08 18:18:25.879810448 +0000 UTC m=+26.631119923" Jul 08 18:18:27.764950 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:27.764696 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:27.765400 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:27.764696 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:27.765400 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:27.765053 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:27.765400 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:27.764703 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:27.765400 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:27.765144 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xw9f2" podUID="506c4790-d442-4464-8b83-06d2891a6df0" Jul 08 18:18:27.765400 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:27.765205 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:28.872829 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:28.872791 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" event={"ID":"a05de7da-b5fb-4f8b-b52f-d9222471fae2","Type":"ContainerStarted","Data":"4222d88bf1b48cedd66865fdfb470fdbe55f2c1481457c0c27182517e221f419"} Jul 08 18:18:28.873603 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:28.873381 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:28.875237 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:28.875077 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmdn8" event={"ID":"33915cb2-f6ca-4540-9425-ff653a28c8a2","Type":"ContainerStarted","Data":"41cc03f321e836a07808533419d053192ea441c5f298dae9613c17ee01f2ed2d"} Jul 08 18:18:28.889449 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:28.889425 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:28.900178 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:28.900131 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" podStartSLOduration=10.361289034 podStartE2EDuration="29.900117527s" totalImagesPullingTime="19.538828493s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:28.89977763 +0000 UTC m=+29.651087121" watchObservedRunningTime="2026-07-08 18:18:28.900117527 +0000 UTC m=+29.651427000" Jul 08 18:18:29.643514 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:29.643476 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:29.742971 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:29.742895 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:29.765448 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:29.765416 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:29.765626 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:29.765517 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:29.765626 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:29.765534 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xw9f2" podUID="506c4790-d442-4464-8b83-06d2891a6df0" Jul 08 18:18:29.765738 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:29.765662 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:29.765738 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:29.765697 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:29.765823 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:29.765776 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:29.879178 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:29.879144 2574 generic.go:356] "Generic (PLEG): container finished" podID="33915cb2-f6ca-4540-9425-ff653a28c8a2" containerID="41cc03f321e836a07808533419d053192ea441c5f298dae9613c17ee01f2ed2d" exitCode=0 Jul 08 18:18:29.879572 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:29.879232 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmdn8" event={"ID":"33915cb2-f6ca-4540-9425-ff653a28c8a2","Type":"ContainerDied","Data":"41cc03f321e836a07808533419d053192ea441c5f298dae9613c17ee01f2ed2d"} Jul 08 18:18:29.879744 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:29.879722 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:29.879909 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:29.879790 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:29.895377 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:29.895306 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:18:30.489796 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:30.489768 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xw9f2"] Jul 08 18:18:30.489982 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:30.489947 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:30.490076 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:30.490048 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xw9f2" podUID="506c4790-d442-4464-8b83-06d2891a6df0" Jul 08 18:18:30.492099 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:30.492062 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fq9lp"] Jul 08 18:18:30.492225 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:30.492202 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:30.492319 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:30.492292 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:30.492815 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:30.492792 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ts8sd"] Jul 08 18:18:30.492956 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:30.492918 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:30.493038 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:30.493017 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:30.643356 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:30.643283 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:30.743682 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:30.743649 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:30.797146 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:30.797107 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:31.765112 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:31.765085 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:31.765539 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:31.765123 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:31.765539 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:31.765216 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:31.765539 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:31.765322 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:31.885464 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:31.885432 2574 generic.go:356] "Generic (PLEG): container finished" podID="33915cb2-f6ca-4540-9425-ff653a28c8a2" containerID="a850c2bef1658377ceb51eb244a36326a6920cec40878c18c22a181c31ac15de" exitCode=0 Jul 08 18:18:31.885592 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:31.885511 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmdn8" event={"ID":"33915cb2-f6ca-4540-9425-ff653a28c8a2","Type":"ContainerDied","Data":"a850c2bef1658377ceb51eb244a36326a6920cec40878c18c22a181c31ac15de"} Jul 08 18:18:32.632762 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:32.632736 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:32.765379 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:32.765345 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:32.765748 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:32.765451 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xw9f2" podUID="506c4790-d442-4464-8b83-06d2891a6df0" Jul 08 18:18:33.356774 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:33.356507 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs\") pod \"network-metrics-daemon-ts8sd\" (UID: \"0d868453-b10b-4dd9-b843-91de3f1b48d3\") " pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:33.356774 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:33.356749 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret\") pod \"global-pull-secret-syncer-xw9f2\" (UID: \"506c4790-d442-4464-8b83-06d2891a6df0\") " pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:33.356977 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:33.356659 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:33.356977 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:33.356817 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:33.356977 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:33.356881 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret podName:506c4790-d442-4464-8b83-06d2891a6df0 nodeName:}" failed. No retries permitted until 2026-07-08 18:19:05.356864877 +0000 UTC m=+66.108174344 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret") pod "global-pull-secret-syncer-xw9f2" (UID: "506c4790-d442-4464-8b83-06d2891a6df0") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:33.356977 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:33.356897 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs podName:0d868453-b10b-4dd9-b843-91de3f1b48d3 nodeName:}" failed. No retries permitted until 2026-07-08 18:19:05.356890249 +0000 UTC m=+66.108199700 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs") pod "network-metrics-daemon-ts8sd" (UID: "0d868453-b10b-4dd9-b843-91de3f1b48d3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:33.457598 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:33.457564 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g5wf\" (UniqueName: \"kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf\") pod \"network-check-target-fq9lp\" (UID: \"3a8d7a1d-49b0-41a0-a297-2898090ea58f\") " pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:33.457760 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:33.457724 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:18:33.457760 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:33.457743 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:18:33.457760 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:33.457753 2574 projected.go:196] Error preparing data for projected volume kube-api-access-6g5wf for pod openshift-network-diagnostics/network-check-target-fq9lp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:33.457882 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:33.457802 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf podName:3a8d7a1d-49b0-41a0-a297-2898090ea58f nodeName:}" failed. No retries permitted until 2026-07-08 18:19:05.457789051 +0000 UTC m=+66.209098502 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6g5wf" (UniqueName: "kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf") pod "network-check-target-fq9lp" (UID: "3a8d7a1d-49b0-41a0-a297-2898090ea58f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:33.764738 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:33.764661 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:33.764888 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:33.764783 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:18:33.764888 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:33.764838 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:33.764979 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:33.764943 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fq9lp" podUID="3a8d7a1d-49b0-41a0-a297-2898090ea58f" Jul 08 18:18:33.891430 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:33.891396 2574 generic.go:356] "Generic (PLEG): container finished" podID="33915cb2-f6ca-4540-9425-ff653a28c8a2" containerID="28c9a4c8e1948ab739842a63eb12840320b0948215567be7ffdf0d5499e97a45" exitCode=0 Jul 08 18:18:33.891786 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:33.891442 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmdn8" event={"ID":"33915cb2-f6ca-4540-9425-ff653a28c8a2","Type":"ContainerDied","Data":"28c9a4c8e1948ab739842a63eb12840320b0948215567be7ffdf0d5499e97a45"} Jul 08 18:18:34.540765 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.540733 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-97.ec2.internal" event="NodeReady" Jul 08 18:18:34.540960 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.540916 2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 18:18:34.577655 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.577626 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75cf5b4d8b-6v75j"] Jul 08 18:18:34.581177 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.581152 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.583682 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.583659 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hm9k9\"" Jul 08 18:18:34.583783 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.583656 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 18:18:34.583783 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.583660 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 18:18:34.584007 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.583983 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 18:18:34.592468 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.592057 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cz6kw"] Jul 08 18:18:34.595749 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.595722 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 18:18:34.596329 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.596306 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9fl2r"] Jul 08 18:18:34.597079 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.596988 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cz6kw" Jul 08 18:18:34.599671 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.599649 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 18:18:34.599778 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.599683 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d4sdr\"" Jul 08 18:18:34.599778 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.599715 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9fl2r" Jul 08 18:18:34.599778 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.599738 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 18:18:34.600042 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.599653 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75cf5b4d8b-6v75j"] Jul 08 18:18:34.602203 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.602160 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9fl2r"] Jul 08 18:18:34.602437 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.602416 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 18:18:34.602545 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.602503 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 18:18:34.602628 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.602609 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 18:18:34.602956 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.602939 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-r6bqm\"" Jul 08 18:18:34.605892 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.605872 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cz6kw"] Jul 08 18:18:34.666501 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.666419 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8fa3a586-75b0-44a3-a133-9d4caa863f22-image-registry-private-configuration\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.666501 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.666457 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-bound-sa-token\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.666501 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.666489 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.666767 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.666567 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-certificates\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.666767 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.666621 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc56c\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-kube-api-access-qc56c\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.666767 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.666689 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8fa3a586-75b0-44a3-a133-9d4caa863f22-trusted-ca\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.666767 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.666734 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8fa3a586-75b0-44a3-a133-9d4caa863f22-ca-trust-extracted\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.666767 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.666758 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8fa3a586-75b0-44a3-a133-9d4caa863f22-installation-pull-secrets\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.715238 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.715182 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:34.765233 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.765203 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:18:34.767245 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.767218 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-certificates\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.767366 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.767261 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmx2w\" (UniqueName: \"kubernetes.io/projected/1e9502fd-4d0e-4186-84b2-76b6db2593b1-kube-api-access-xmx2w\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:18:34.767366 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.767301 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e9502fd-4d0e-4186-84b2-76b6db2593b1-tmp-dir\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:18:34.767366 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.767331 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc56c\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-kube-api-access-qc56c\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.767366 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.767360 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58fbd0df-e933-4bee-9916-4b786b0205a8-cert\") pod \"ingress-canary-9fl2r\" (UID: \"58fbd0df-e933-4bee-9916-4b786b0205a8\") " pod="openshift-ingress-canary/ingress-canary-9fl2r" Jul 08 18:18:34.767569 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.767383 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8fa3a586-75b0-44a3-a133-9d4caa863f22-trusted-ca\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.767569 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.767410 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1e9502fd-4d0e-4186-84b2-76b6db2593b1-config-volume\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:18:34.767569 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.767435 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs228\" (UniqueName: \"kubernetes.io/projected/58fbd0df-e933-4bee-9916-4b786b0205a8-kube-api-access-cs228\") pod \"ingress-canary-9fl2r\" (UID: \"58fbd0df-e933-4bee-9916-4b786b0205a8\") " pod="openshift-ingress-canary/ingress-canary-9fl2r" Jul 08 18:18:34.767569 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.767461 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8fa3a586-75b0-44a3-a133-9d4caa863f22-ca-trust-extracted\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.767569 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.767475 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8fa3a586-75b0-44a3-a133-9d4caa863f22-installation-pull-secrets\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.767569 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.767489 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:18:34.767569 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.767505 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8fa3a586-75b0-44a3-a133-9d4caa863f22-image-registry-private-configuration\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.767569 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.767520 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-bound-sa-token\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.767569 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.767547 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.768011 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:34.767659 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:18:34.768011 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:34.767673 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cf5b4d8b-6v75j: secret "image-registry-tls" not found Jul 08 18:18:34.768011 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:34.767738 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls podName:8fa3a586-75b0-44a3-a133-9d4caa863f22 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:35.267718735 +0000 UTC m=+36.019028202 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls") pod "image-registry-75cf5b4d8b-6v75j" (UID: "8fa3a586-75b0-44a3-a133-9d4caa863f22") : secret "image-registry-tls" not found Jul 08 18:18:34.768694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.768279 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8fa3a586-75b0-44a3-a133-9d4caa863f22-ca-trust-extracted\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.768694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.768418 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-certificates\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.768860 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.768724 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 18:18:34.769411 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.769159 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8fa3a586-75b0-44a3-a133-9d4caa863f22-trusted-ca\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.772441 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.772422 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8fa3a586-75b0-44a3-a133-9d4caa863f22-installation-pull-secrets\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.772512 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.772442 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8fa3a586-75b0-44a3-a133-9d4caa863f22-image-registry-private-configuration\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.777942 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.777919 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-bound-sa-token\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.778095 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.778079 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc56c\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-kube-api-access-qc56c\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:34.817512 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.817479 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:34.868510 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.868479 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmx2w\" (UniqueName: \"kubernetes.io/projected/1e9502fd-4d0e-4186-84b2-76b6db2593b1-kube-api-access-xmx2w\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:18:34.868694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.868535 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e9502fd-4d0e-4186-84b2-76b6db2593b1-tmp-dir\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:18:34.868694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.868577 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58fbd0df-e933-4bee-9916-4b786b0205a8-cert\") pod \"ingress-canary-9fl2r\" (UID: \"58fbd0df-e933-4bee-9916-4b786b0205a8\") " pod="openshift-ingress-canary/ingress-canary-9fl2r" Jul 08 18:18:34.868694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.868619 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1e9502fd-4d0e-4186-84b2-76b6db2593b1-config-volume\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:18:34.868694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.868646 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs228\" (UniqueName: \"kubernetes.io/projected/58fbd0df-e933-4bee-9916-4b786b0205a8-kube-api-access-cs228\") pod \"ingress-canary-9fl2r\" (UID: \"58fbd0df-e933-4bee-9916-4b786b0205a8\") " pod="openshift-ingress-canary/ingress-canary-9fl2r" Jul 08 18:18:34.868694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.868671 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:18:34.868954 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:34.868805 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:18:34.868954 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:34.868906 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls podName:1e9502fd-4d0e-4186-84b2-76b6db2593b1 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:35.368885068 +0000 UTC m=+36.120194538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls") pod "dns-default-cz6kw" (UID: "1e9502fd-4d0e-4186-84b2-76b6db2593b1") : secret "dns-default-metrics-tls" not found Jul 08 18:18:34.869230 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.869206 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e9502fd-4d0e-4186-84b2-76b6db2593b1-tmp-dir\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:18:34.869493 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.869476 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1e9502fd-4d0e-4186-84b2-76b6db2593b1-config-volume\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:18:34.871352 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.871320 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58fbd0df-e933-4bee-9916-4b786b0205a8-cert\") pod \"ingress-canary-9fl2r\" (UID: \"58fbd0df-e933-4bee-9916-4b786b0205a8\") " pod="openshift-ingress-canary/ingress-canary-9fl2r" Jul 08 18:18:34.879652 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.879632 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs228\" (UniqueName: \"kubernetes.io/projected/58fbd0df-e933-4bee-9916-4b786b0205a8-kube-api-access-cs228\") pod \"ingress-canary-9fl2r\" (UID: \"58fbd0df-e933-4bee-9916-4b786b0205a8\") " pod="openshift-ingress-canary/ingress-canary-9fl2r" Jul 08 18:18:34.879652 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.879643 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmx2w\" (UniqueName: \"kubernetes.io/projected/1e9502fd-4d0e-4186-84b2-76b6db2593b1-kube-api-access-xmx2w\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:18:34.916480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.916447 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9fl2r" Jul 08 18:18:34.969499 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:34.969463 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:35.076439 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:35.076422 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9fl2r"] Jul 08 18:18:35.080050 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:18:35.080028 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58fbd0df_e933_4bee_9916_4b786b0205a8.slice/crio-31e2e1218aaed0aef4b4b224913336bff0d5e9facf86763912eb539044b1d234 WatchSource:0}: Error finding container 31e2e1218aaed0aef4b4b224913336bff0d5e9facf86763912eb539044b1d234: Status 404 returned error can't find the container with id 31e2e1218aaed0aef4b4b224913336bff0d5e9facf86763912eb539044b1d234 Jul 08 18:18:35.271302 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:35.271211 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:35.271462 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:35.271349 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:18:35.271462 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:35.271366 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cf5b4d8b-6v75j: secret "image-registry-tls" not found Jul 08 18:18:35.271462 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:35.271420 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls podName:8fa3a586-75b0-44a3-a133-9d4caa863f22 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:36.271403888 +0000 UTC m=+37.022713344 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls") pod "image-registry-75cf5b4d8b-6v75j" (UID: "8fa3a586-75b0-44a3-a133-9d4caa863f22") : secret "image-registry-tls" not found Jul 08 18:18:35.372431 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:35.372397 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:18:35.372594 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:35.372570 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:18:35.372666 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:35.372646 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls podName:1e9502fd-4d0e-4186-84b2-76b6db2593b1 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:36.372623448 +0000 UTC m=+37.123932903 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls") pod "dns-default-cz6kw" (UID: "1e9502fd-4d0e-4186-84b2-76b6db2593b1") : secret "dns-default-metrics-tls" not found Jul 08 18:18:35.765241 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:35.765157 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:18:35.765410 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:35.765156 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:18:35.768291 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:35.767902 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 18:18:35.768291 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:35.768034 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 18:18:35.768291 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:35.768147 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 18:18:35.768291 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:35.768195 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9qsc4\"" Jul 08 18:18:35.768603 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:35.768578 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kcpkw\"" Jul 08 18:18:35.898584 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:35.898546 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9fl2r" event={"ID":"58fbd0df-e933-4bee-9916-4b786b0205a8","Type":"ContainerStarted","Data":"31e2e1218aaed0aef4b4b224913336bff0d5e9facf86763912eb539044b1d234"} Jul 08 18:18:36.281227 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:36.281189 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:36.281933 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:36.281329 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:18:36.281933 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:36.281349 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cf5b4d8b-6v75j: secret "image-registry-tls" not found Jul 08 18:18:36.281933 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:36.281411 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls podName:8fa3a586-75b0-44a3-a133-9d4caa863f22 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:38.281391727 +0000 UTC m=+39.032701184 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls") pod "image-registry-75cf5b4d8b-6v75j" (UID: "8fa3a586-75b0-44a3-a133-9d4caa863f22") : secret "image-registry-tls" not found Jul 08 18:18:36.382589 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:36.382548 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:18:36.382735 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:36.382674 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:18:36.382735 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:36.382729 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls podName:1e9502fd-4d0e-4186-84b2-76b6db2593b1 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:38.382714548 +0000 UTC m=+39.134024000 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls") pod "dns-default-cz6kw" (UID: "1e9502fd-4d0e-4186-84b2-76b6db2593b1") : secret "dns-default-metrics-tls" not found Jul 08 18:18:38.296170 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:38.295923 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:38.296575 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:38.296106 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:18:38.296575 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:38.296211 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cf5b4d8b-6v75j: secret "image-registry-tls" not found Jul 08 18:18:38.296575 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:38.296271 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls podName:8fa3a586-75b0-44a3-a133-9d4caa863f22 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:42.296250979 +0000 UTC m=+43.047560430 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls") pod "image-registry-75cf5b4d8b-6v75j" (UID: "8fa3a586-75b0-44a3-a133-9d4caa863f22") : secret "image-registry-tls" not found Jul 08 18:18:38.397131 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:38.397095 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:18:38.397295 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:38.397266 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:18:38.397347 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:38.397335 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls podName:1e9502fd-4d0e-4186-84b2-76b6db2593b1 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:42.397319535 +0000 UTC m=+43.148628990 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls") pod "dns-default-cz6kw" (UID: "1e9502fd-4d0e-4186-84b2-76b6db2593b1") : secret "dns-default-metrics-tls" not found Jul 08 18:18:39.332912 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:39.332883 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:39.910605 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:39.910577 2574 generic.go:356] "Generic (PLEG): container finished" podID="33915cb2-f6ca-4540-9425-ff653a28c8a2" containerID="2e4789fb2d31d3856594d32742ed81405f5cf4f42a02bdc0c3b713ac10dfdd97" exitCode=0 Jul 08 18:18:39.910763 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:39.910650 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmdn8" event={"ID":"33915cb2-f6ca-4540-9425-ff653a28c8a2","Type":"ContainerDied","Data":"2e4789fb2d31d3856594d32742ed81405f5cf4f42a02bdc0c3b713ac10dfdd97"} Jul 08 18:18:39.912033 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:39.912010 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9fl2r" event={"ID":"58fbd0df-e933-4bee-9916-4b786b0205a8","Type":"ContainerStarted","Data":"36f5c0882eb423b06e3933d9ed37b7cb507ec392d9edb9c7b770a74201fa17f8"} Jul 08 18:18:39.950444 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:39.950393 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9fl2r" podStartSLOduration=1.798907883 podStartE2EDuration="5.950374471s" totalImagesPullingTime="4.151466588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:39.949988407 +0000 UTC m=+40.701297883" watchObservedRunningTime="2026-07-08 18:18:39.950374471 +0000 UTC m=+40.701683948" Jul 08 18:18:40.917485 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:40.917452 2574 generic.go:356] "Generic (PLEG): container finished" podID="33915cb2-f6ca-4540-9425-ff653a28c8a2" containerID="9ba337a018bb5ee641a8c821b20095f2773153daf672839df42d2d714eeb7495" exitCode=0 Jul 08 18:18:40.918066 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:40.917537 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmdn8" event={"ID":"33915cb2-f6ca-4540-9425-ff653a28c8a2","Type":"ContainerDied","Data":"9ba337a018bb5ee641a8c821b20095f2773153daf672839df42d2d714eeb7495"} Jul 08 18:18:41.697355 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:41.697320 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:41.771348 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:41.771312 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:41.922998 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:41.922966 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tmdn8" event={"ID":"33915cb2-f6ca-4540-9425-ff653a28c8a2","Type":"ContainerStarted","Data":"6b488786cda6dc3b18ad2150ff97ae4e70d57ae1b975cb89dd03c297b4b63853"} Jul 08 18:18:42.068237 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:42.068199 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:42.169100 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:42.169064 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:42.327516 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:42.327437 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:42.327652 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:42.327557 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:18:42.327652 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:42.327570 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cf5b4d8b-6v75j: secret "image-registry-tls" not found Jul 08 18:18:42.327740 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:42.327654 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls podName:8fa3a586-75b0-44a3-a133-9d4caa863f22 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:50.327639742 +0000 UTC m=+51.078949194 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls") pod "image-registry-75cf5b4d8b-6v75j" (UID: "8fa3a586-75b0-44a3-a133-9d4caa863f22") : secret "image-registry-tls" not found Jul 08 18:18:42.428302 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:42.428269 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:18:42.428455 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:42.428377 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:18:42.428455 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:42.428424 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls podName:1e9502fd-4d0e-4186-84b2-76b6db2593b1 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:50.428411827 +0000 UTC m=+51.179721279 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls") pod "dns-default-cz6kw" (UID: "1e9502fd-4d0e-4186-84b2-76b6db2593b1") : secret "dns-default-metrics-tls" not found Jul 08 18:18:44.258810 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:44.258780 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:44.315994 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:44.315960 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:45.726764 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:45.726735 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:45.827046 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:45.827015 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:45.885175 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:45.885145 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:46.861403 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:46.861365 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:46.917540 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:46.917504 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:49.486030 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:49.485978 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tmdn8" podStartSLOduration=17.598120739 podStartE2EDuration="50.485960127s" totalImagesPullingTime="32.887839388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:17:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:41.948894273 +0000 UTC m=+42.700203748" watchObservedRunningTime="2026-07-08 18:18:49.485960127 +0000 UTC m=+50.237269579" Jul 08 18:18:49.486399 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:49.486164 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db7cf857d-m7qxj"] Jul 08 18:18:49.502914 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:49.502883 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db7cf857d-m7qxj"] Jul 08 18:18:49.503076 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:49.502948 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db7cf857d-m7qxj" Jul 08 18:18:49.506450 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:49.506426 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-965st\"" Jul 08 18:18:49.506563 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:49.506473 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 08 18:18:49.506563 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:49.506432 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 08 18:18:49.506563 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:49.506436 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 08 18:18:49.506716 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:49.506432 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 08 18:18:49.582540 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:49.582513 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/39496412-e867-4e46-b4d3-45811fa0eb2c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5db7cf857d-m7qxj\" (UID: \"39496412-e867-4e46-b4d3-45811fa0eb2c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db7cf857d-m7qxj" Jul 08 18:18:49.582540 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:49.582544 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sj9q\" (UniqueName: \"kubernetes.io/projected/39496412-e867-4e46-b4d3-45811fa0eb2c-kube-api-access-5sj9q\") pod \"managed-serviceaccount-addon-agent-5db7cf857d-m7qxj\" (UID: \"39496412-e867-4e46-b4d3-45811fa0eb2c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db7cf857d-m7qxj" Jul 08 18:18:49.683279 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:49.683249 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/39496412-e867-4e46-b4d3-45811fa0eb2c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5db7cf857d-m7qxj\" (UID: \"39496412-e867-4e46-b4d3-45811fa0eb2c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db7cf857d-m7qxj" Jul 08 18:18:49.683279 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:49.683282 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sj9q\" (UniqueName: \"kubernetes.io/projected/39496412-e867-4e46-b4d3-45811fa0eb2c-kube-api-access-5sj9q\") pod \"managed-serviceaccount-addon-agent-5db7cf857d-m7qxj\" (UID: \"39496412-e867-4e46-b4d3-45811fa0eb2c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db7cf857d-m7qxj" Jul 08 18:18:49.686705 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:49.686670 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/39496412-e867-4e46-b4d3-45811fa0eb2c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5db7cf857d-m7qxj\" (UID: \"39496412-e867-4e46-b4d3-45811fa0eb2c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db7cf857d-m7qxj" Jul 08 18:18:49.691467 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:49.691442 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sj9q\" (UniqueName: \"kubernetes.io/projected/39496412-e867-4e46-b4d3-45811fa0eb2c-kube-api-access-5sj9q\") pod \"managed-serviceaccount-addon-agent-5db7cf857d-m7qxj\" (UID: \"39496412-e867-4e46-b4d3-45811fa0eb2c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db7cf857d-m7qxj" Jul 08 18:18:49.820157 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:49.820133 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db7cf857d-m7qxj" Jul 08 18:18:49.963411 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:49.963382 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db7cf857d-m7qxj"] Jul 08 18:18:50.387542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:50.387507 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:18:50.387706 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:50.387663 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:18:50.387706 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:50.387677 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cf5b4d8b-6v75j: secret "image-registry-tls" not found Jul 08 18:18:50.387777 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:50.387735 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls podName:8fa3a586-75b0-44a3-a133-9d4caa863f22 nodeName:}" failed. No retries permitted until 2026-07-08 18:19:06.387718645 +0000 UTC m=+67.139028098 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls") pod "image-registry-75cf5b4d8b-6v75j" (UID: "8fa3a586-75b0-44a3-a133-9d4caa863f22") : secret "image-registry-tls" not found Jul 08 18:18:50.488443 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:50.488400 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:18:50.488941 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:50.488580 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:18:50.488941 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:18:50.488667 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls podName:1e9502fd-4d0e-4186-84b2-76b6db2593b1 nodeName:}" failed. No retries permitted until 2026-07-08 18:19:06.488645206 +0000 UTC m=+67.239954659 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls") pod "dns-default-cz6kw" (UID: "1e9502fd-4d0e-4186-84b2-76b6db2593b1") : secret "dns-default-metrics-tls" not found Jul 08 18:18:50.946095 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:50.946034 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db7cf857d-m7qxj" event={"ID":"39496412-e867-4e46-b4d3-45811fa0eb2c","Type":"ContainerStarted","Data":"e9adf9316552265a899079aee909bcd062642270dd448e50dfbe55def26619ca"} Jul 08 18:18:53.955898 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:53.955860 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db7cf857d-m7qxj" event={"ID":"39496412-e867-4e46-b4d3-45811fa0eb2c","Type":"ContainerStarted","Data":"ccb779ac5b059bf93f0366896fdc8d9014692afd535b7cca7bfbe2efa512be2b"} Jul 08 18:18:53.972191 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:18:53.972146 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db7cf857d-m7qxj" podStartSLOduration=1.72259731 podStartE2EDuration="4.972132483s" totalImagesPullingTime="3.249535173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:53.971545667 +0000 UTC m=+54.722855141" watchObservedRunningTime="2026-07-08 18:18:53.972132483 +0000 UTC m=+54.723441969" Jul 08 18:19:01.896918 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:01.896889 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x56pf" Jul 08 18:19:02.052439 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:02.052410 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:19:02.123549 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:02.123514 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:19:05.399234 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:05.399189 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs\") pod \"network-metrics-daemon-ts8sd\" (UID: \"0d868453-b10b-4dd9-b843-91de3f1b48d3\") " pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:19:05.399234 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:05.399241 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret\") pod \"global-pull-secret-syncer-xw9f2\" (UID: \"506c4790-d442-4464-8b83-06d2891a6df0\") " pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:19:05.402288 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:05.402269 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 18:19:05.402360 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:05.402301 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 18:19:05.410272 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:19:05.410249 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 18:19:05.410344 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:19:05.410306 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs podName:0d868453-b10b-4dd9-b843-91de3f1b48d3 nodeName:}" failed. No retries permitted until 2026-07-08 18:20:09.410291064 +0000 UTC m=+130.161600515 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs") pod "network-metrics-daemon-ts8sd" (UID: "0d868453-b10b-4dd9-b843-91de3f1b48d3") : secret "metrics-daemon-secret" not found Jul 08 18:19:05.412464 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:05.412446 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/506c4790-d442-4464-8b83-06d2891a6df0-original-pull-secret\") pod \"global-pull-secret-syncer-xw9f2\" (UID: \"506c4790-d442-4464-8b83-06d2891a6df0\") " pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:19:05.500327 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:05.500288 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g5wf\" (UniqueName: \"kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf\") pod \"network-check-target-fq9lp\" (UID: \"3a8d7a1d-49b0-41a0-a297-2898090ea58f\") " pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:19:05.503120 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:05.503091 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 18:19:05.513655 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:05.513629 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 18:19:05.523967 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:05.523940 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g5wf\" (UniqueName: \"kubernetes.io/projected/3a8d7a1d-49b0-41a0-a297-2898090ea58f-kube-api-access-6g5wf\") pod \"network-check-target-fq9lp\" (UID: \"3a8d7a1d-49b0-41a0-a297-2898090ea58f\") " pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:19:05.690891 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:05.690774 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xw9f2" Jul 08 18:19:05.792079 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:05.792052 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kcpkw\"" Jul 08 18:19:05.800320 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:05.800297 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:19:05.810416 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:05.810349 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xw9f2"] Jul 08 18:19:05.813994 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:19:05.813967 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod506c4790_d442_4464_8b83_06d2891a6df0.slice/crio-126fcc4cb1b3539d2b79326d8ddbca8000905eb2b2bcb6f6768ff3f1e29e7aad WatchSource:0}: Error finding container 126fcc4cb1b3539d2b79326d8ddbca8000905eb2b2bcb6f6768ff3f1e29e7aad: Status 404 returned error can't find the container with id 126fcc4cb1b3539d2b79326d8ddbca8000905eb2b2bcb6f6768ff3f1e29e7aad Jul 08 18:19:05.927388 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:05.927357 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fq9lp"] Jul 08 18:19:05.931007 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:19:05.930978 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a8d7a1d_49b0_41a0_a297_2898090ea58f.slice/crio-8fc70343ff28454486e00bfbd726ed83fdd7862162d4e7c453c48aac566b6993 WatchSource:0}: Error finding container 8fc70343ff28454486e00bfbd726ed83fdd7862162d4e7c453c48aac566b6993: Status 404 returned error can't find the container with id 8fc70343ff28454486e00bfbd726ed83fdd7862162d4e7c453c48aac566b6993 Jul 08 18:19:05.986920 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:05.986836 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xw9f2" event={"ID":"506c4790-d442-4464-8b83-06d2891a6df0","Type":"ContainerStarted","Data":"126fcc4cb1b3539d2b79326d8ddbca8000905eb2b2bcb6f6768ff3f1e29e7aad"} Jul 08 18:19:05.988011 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:05.987985 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fq9lp" event={"ID":"3a8d7a1d-49b0-41a0-a297-2898090ea58f","Type":"ContainerStarted","Data":"8fc70343ff28454486e00bfbd726ed83fdd7862162d4e7c453c48aac566b6993"} Jul 08 18:19:06.408000 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:06.407964 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:19:06.408475 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:19:06.408144 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:19:06.408475 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:19:06.408161 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cf5b4d8b-6v75j: secret "image-registry-tls" not found Jul 08 18:19:06.408475 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:19:06.408227 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls podName:8fa3a586-75b0-44a3-a133-9d4caa863f22 nodeName:}" failed. No retries permitted until 2026-07-08 18:19:38.408208428 +0000 UTC m=+99.159517897 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls") pod "image-registry-75cf5b4d8b-6v75j" (UID: "8fa3a586-75b0-44a3-a133-9d4caa863f22") : secret "image-registry-tls" not found Jul 08 18:19:06.508868 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:06.508818 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:19:06.509026 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:19:06.508987 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:19:06.509069 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:19:06.509048 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls podName:1e9502fd-4d0e-4186-84b2-76b6db2593b1 nodeName:}" failed. No retries permitted until 2026-07-08 18:19:38.509033157 +0000 UTC m=+99.260342609 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls") pod "dns-default-cz6kw" (UID: "1e9502fd-4d0e-4186-84b2-76b6db2593b1") : secret "dns-default-metrics-tls" not found Jul 08 18:19:08.736927 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:08.736817 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:19:08.813454 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:08.813414 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:19:12.006479 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:12.006434 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xw9f2" event={"ID":"506c4790-d442-4464-8b83-06d2891a6df0","Type":"ContainerStarted","Data":"a3f0b48fdd211c90bc45a1dd20dc7dfcddb49a5df1ca8131a8c93994d60cc2d0"} Jul 08 18:19:12.007842 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:12.007817 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fq9lp" event={"ID":"3a8d7a1d-49b0-41a0-a297-2898090ea58f","Type":"ContainerStarted","Data":"ddcecd82f3d839f73aa3d6cbb3ab0c0e3648df5e5d3c59ad301163b0e301cbcf"} Jul 08 18:19:12.007984 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:12.007922 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:19:12.021474 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:12.021427 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xw9f2" podStartSLOduration=67.836297296 podStartE2EDuration="1m13.02141166s" totalImagesPullingTime="5.185114364s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:19:12.020828867 +0000 UTC m=+72.772138340" watchObservedRunningTime="2026-07-08 18:19:12.02141166 +0000 UTC m=+72.772721136" Jul 08 18:19:12.036779 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:12.036725 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fq9lp" podStartSLOduration=67.972664463 podStartE2EDuration="1m13.036711348s" totalImagesPullingTime="5.064046885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:19:12.036030559 +0000 UTC m=+72.787340037" watchObservedRunningTime="2026-07-08 18:19:12.036711348 +0000 UTC m=+72.788020799" Jul 08 18:19:33.044050 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:33.043908 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:19:33.102307 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:33.102262 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:19:38.439958 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:38.439903 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:19:38.440348 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:19:38.440052 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:19:38.440348 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:19:38.440072 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cf5b4d8b-6v75j: secret "image-registry-tls" not found Jul 08 18:19:38.440348 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:19:38.440136 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls podName:8fa3a586-75b0-44a3-a133-9d4caa863f22 nodeName:}" failed. No retries permitted until 2026-07-08 18:20:42.440121004 +0000 UTC m=+163.191430456 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls") pod "image-registry-75cf5b4d8b-6v75j" (UID: "8fa3a586-75b0-44a3-a133-9d4caa863f22") : secret "image-registry-tls" not found Jul 08 18:19:38.540252 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:38.540215 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:19:38.540414 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:19:38.540362 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:19:38.540460 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:19:38.540433 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls podName:1e9502fd-4d0e-4186-84b2-76b6db2593b1 nodeName:}" failed. No retries permitted until 2026-07-08 18:20:42.540417952 +0000 UTC m=+163.291727403 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls") pod "dns-default-cz6kw" (UID: "1e9502fd-4d0e-4186-84b2-76b6db2593b1") : secret "dns-default-metrics-tls" not found Jul 08 18:19:43.011683 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:19:43.011656 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fq9lp" Jul 08 18:20:09.450696 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:20:09.450644 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs\") pod \"network-metrics-daemon-ts8sd\" (UID: \"0d868453-b10b-4dd9-b843-91de3f1b48d3\") " pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:20:09.451222 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:20:09.450795 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 18:20:09.451222 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:20:09.450878 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs podName:0d868453-b10b-4dd9-b843-91de3f1b48d3 nodeName:}" failed. No retries permitted until 2026-07-08 18:22:11.450842638 +0000 UTC m=+252.202152091 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs") pod "network-metrics-daemon-ts8sd" (UID: "0d868453-b10b-4dd9-b843-91de3f1b48d3") : secret "metrics-daemon-secret" not found Jul 08 18:20:37.597317 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:20:37.597270 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" podUID="8fa3a586-75b0-44a3-a133-9d4caa863f22" Jul 08 18:20:37.609708 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:20:37.609688 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-cz6kw" podUID="1e9502fd-4d0e-4186-84b2-76b6db2593b1" Jul 08 18:20:38.211553 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:20:38.211526 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:20:38.211721 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:20:38.211526 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cz6kw" Jul 08 18:20:38.784133 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:20:38.784081 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:20:42.475785 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:20:42.475738 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls\") pod \"image-registry-75cf5b4d8b-6v75j\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:20:42.476179 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:20:42.475924 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:20:42.476179 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:20:42.475946 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cf5b4d8b-6v75j: secret "image-registry-tls" not found Jul 08 18:20:42.476179 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:20:42.476016 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls podName:8fa3a586-75b0-44a3-a133-9d4caa863f22 nodeName:}" failed. No retries permitted until 2026-07-08 18:22:44.475999803 +0000 UTC m=+285.227309254 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls") pod "image-registry-75cf5b4d8b-6v75j" (UID: "8fa3a586-75b0-44a3-a133-9d4caa863f22") : secret "image-registry-tls" not found Jul 08 18:20:42.576549 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:20:42.576497 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:20:42.576715 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:20:42.576643 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:20:42.576759 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:20:42.576732 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls podName:1e9502fd-4d0e-4186-84b2-76b6db2593b1 nodeName:}" failed. No retries permitted until 2026-07-08 18:22:44.576715689 +0000 UTC m=+285.328025142 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls") pod "dns-default-cz6kw" (UID: "1e9502fd-4d0e-4186-84b2-76b6db2593b1") : secret "dns-default-metrics-tls" not found Jul 08 18:20:49.765696 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:20:49.765660 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:20:54.249488 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:20:54.249451 2574 generic.go:356] "Generic (PLEG): container finished" podID="39496412-e867-4e46-b4d3-45811fa0eb2c" containerID="ccb779ac5b059bf93f0366896fdc8d9014692afd535b7cca7bfbe2efa512be2b" exitCode=255 Jul 08 18:20:54.249877 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:20:54.249495 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db7cf857d-m7qxj" event={"ID":"39496412-e867-4e46-b4d3-45811fa0eb2c","Type":"ContainerDied","Data":"ccb779ac5b059bf93f0366896fdc8d9014692afd535b7cca7bfbe2efa512be2b"} Jul 08 18:20:54.254918 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:20:54.254897 2574 scope.go:117] "RemoveContainer" containerID="ccb779ac5b059bf93f0366896fdc8d9014692afd535b7cca7bfbe2efa512be2b" Jul 08 18:20:55.253966 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:20:55.253931 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5db7cf857d-m7qxj" event={"ID":"39496412-e867-4e46-b4d3-45811fa0eb2c","Type":"ContainerStarted","Data":"6e60b79f7464f2910f9bdd8c24bb40b50438d9681312d1aa55d95e8af926d579"} Jul 08 18:21:41.486115 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:21:41.486080 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:21:52.823773 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:21:52.823741 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:11.494396 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:11.494357 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs\") pod \"network-metrics-daemon-ts8sd\" (UID: \"0d868453-b10b-4dd9-b843-91de3f1b48d3\") " pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:22:11.494902 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:11.494506 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 18:22:11.494902 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:11.494569 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs podName:0d868453-b10b-4dd9-b843-91de3f1b48d3 nodeName:}" failed. No retries permitted until 2026-07-08 18:24:13.494552733 +0000 UTC m=+374.245862185 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs") pod "network-metrics-daemon-ts8sd" (UID: "0d868453-b10b-4dd9-b843-91de3f1b48d3") : secret "metrics-daemon-secret" not found Jul 08 18:22:13.652923 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.652889 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-qkmvz"] Jul 08 18:22:13.655743 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.655728 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph"] Jul 08 18:22:13.655880 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.655861 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.658368 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.658346 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-76f778dfc5-nqd5b"] Jul 08 18:22:13.658518 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.658470 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" Jul 08 18:22:13.661052 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.661032 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:13.661206 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.661186 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 08 18:22:13.661369 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.661137 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 08 18:22:13.661452 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.661136 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 18:22:13.661521 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.661189 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 18:22:13.663294 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.663272 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-4ljlr\"" Jul 08 18:22:13.663394 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.663275 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 08 18:22:13.663394 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.663367 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 08 18:22:13.663498 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.663438 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 18:22:13.663498 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.663469 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-qxcz2\"" Jul 08 18:22:13.663594 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.663565 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 18:22:13.666399 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.666377 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 08 18:22:13.666493 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.666423 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-f4fk8\"" Jul 08 18:22:13.666559 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.666488 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 18:22:13.666559 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.666501 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 18:22:13.666660 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.666557 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 08 18:22:13.666660 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.666604 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 08 18:22:13.666660 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.666637 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 08 18:22:13.676691 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.676671 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph"] Jul 08 18:22:13.676821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.676798 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 08 18:22:13.687475 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.687456 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-qkmvz"] Jul 08 18:22:13.688621 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.688602 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-76f778dfc5-nqd5b"] Jul 08 18:22:13.741182 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.741156 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np"] Jul 08 18:22:13.743927 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.743913 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np" Jul 08 18:22:13.747094 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.747074 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:22:13.747195 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.747100 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 08 18:22:13.747195 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.747074 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 08 18:22:13.747361 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.747348 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 08 18:22:13.747400 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.747382 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-zvkzg\"" Jul 08 18:22:13.754425 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.754404 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np"] Jul 08 18:22:13.808689 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.808666 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:13.808834 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.808693 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-8bsph\" (UID: \"b76b8c21-85aa-43de-b1f7-4224e372467a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" Jul 08 18:22:13.808834 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.808712 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvphq\" (UniqueName: \"kubernetes.io/projected/b76b8c21-85aa-43de-b1f7-4224e372467a-kube-api-access-bvphq\") pod \"cluster-monitoring-operator-64786d9cbb-8bsph\" (UID: \"b76b8c21-85aa-43de-b1f7-4224e372467a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" Jul 08 18:22:13.808834 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.808728 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-tmp\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.808834 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.808746 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-snapshots\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.808834 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.808786 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-serving-cert\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.808834 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.808827 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jj7r\" (UniqueName: \"kubernetes.io/projected/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-kube-api-access-2jj7r\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.809077 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.808908 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-default-certificate\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:13.809077 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.808940 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-stats-auth\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:13.809077 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.808962 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.809077 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.809001 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:13.809077 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.809037 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b76b8c21-85aa-43de-b1f7-4224e372467a-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-8bsph\" (UID: \"b76b8c21-85aa-43de-b1f7-4224e372467a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" Jul 08 18:22:13.809077 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.809068 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.809270 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.809103 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql2qz\" (UniqueName: \"kubernetes.io/projected/8b408e1d-752d-4537-bbfb-145a3520850f-kube-api-access-ql2qz\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:13.834917 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.834888 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:13.909715 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.909640 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:13.909715 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.909682 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-8bsph\" (UID: \"b76b8c21-85aa-43de-b1f7-4224e372467a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" Jul 08 18:22:13.909715 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.909699 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvphq\" (UniqueName: \"kubernetes.io/projected/b76b8c21-85aa-43de-b1f7-4224e372467a-kube-api-access-bvphq\") pod \"cluster-monitoring-operator-64786d9cbb-8bsph\" (UID: \"b76b8c21-85aa-43de-b1f7-4224e372467a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" Jul 08 18:22:13.909715 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.909719 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-tmp\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.910021 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.909744 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fefa4c54-c8d3-48ee-97fd-6358949e39cf-config\") pod \"service-ca-operator-5d46d9ffd4-t58np\" (UID: \"fefa4c54-c8d3-48ee-97fd-6358949e39cf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np" Jul 08 18:22:13.910021 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.909771 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-snapshots\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.910021 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:13.909798 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 18:22:13.910021 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:13.909805 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 18:22:13.910021 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:13.909881 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs podName:8b408e1d-752d-4537-bbfb-145a3520850f nodeName:}" failed. No retries permitted until 2026-07-08 18:22:14.409840959 +0000 UTC m=+255.161150433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs") pod "router-default-76f778dfc5-nqd5b" (UID: "8b408e1d-752d-4537-bbfb-145a3520850f") : secret "router-metrics-certs-default" not found Jul 08 18:22:13.910021 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:13.909897 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls podName:b76b8c21-85aa-43de-b1f7-4224e372467a nodeName:}" failed. No retries permitted until 2026-07-08 18:22:14.409891075 +0000 UTC m=+255.161200527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-8bsph" (UID: "b76b8c21-85aa-43de-b1f7-4224e372467a") : secret "cluster-monitoring-operator-tls" not found Jul 08 18:22:13.910021 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.909928 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-serving-cert\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.910021 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.909964 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jj7r\" (UniqueName: \"kubernetes.io/projected/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-kube-api-access-2jj7r\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.910021 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.910000 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-default-certificate\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:13.910021 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.910022 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-stats-auth\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:13.910526 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.910047 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.910526 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.910084 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:13.910526 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.910125 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fefa4c54-c8d3-48ee-97fd-6358949e39cf-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-t58np\" (UID: \"fefa4c54-c8d3-48ee-97fd-6358949e39cf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np" Jul 08 18:22:13.910526 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.910159 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-tmp\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.910526 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.910163 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b76b8c21-85aa-43de-b1f7-4224e372467a-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-8bsph\" (UID: \"b76b8c21-85aa-43de-b1f7-4224e372467a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" Jul 08 18:22:13.910526 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.910226 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.910526 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.910281 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql2qz\" (UniqueName: \"kubernetes.io/projected/8b408e1d-752d-4537-bbfb-145a3520850f-kube-api-access-ql2qz\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:13.910526 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.910311 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vv2l\" (UniqueName: \"kubernetes.io/projected/fefa4c54-c8d3-48ee-97fd-6358949e39cf-kube-api-access-6vv2l\") pod \"service-ca-operator-5d46d9ffd4-t58np\" (UID: \"fefa4c54-c8d3-48ee-97fd-6358949e39cf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np" Jul 08 18:22:13.910526 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.910483 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-snapshots\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.911009 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:13.910624 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle podName:8b408e1d-752d-4537-bbfb-145a3520850f nodeName:}" failed. No retries permitted until 2026-07-08 18:22:14.410605796 +0000 UTC m=+255.161915287 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle") pod "router-default-76f778dfc5-nqd5b" (UID: "8b408e1d-752d-4537-bbfb-145a3520850f") : configmap references non-existent config key: service-ca.crt Jul 08 18:22:13.911009 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.910907 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b76b8c21-85aa-43de-b1f7-4224e372467a-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-8bsph\" (UID: \"b76b8c21-85aa-43de-b1f7-4224e372467a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" Jul 08 18:22:13.911009 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.910918 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.911400 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.911376 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.912602 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.912585 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-serving-cert\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.912686 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.912633 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-default-certificate\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:13.912825 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.912809 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-stats-auth\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:13.918379 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.918347 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvphq\" (UniqueName: \"kubernetes.io/projected/b76b8c21-85aa-43de-b1f7-4224e372467a-kube-api-access-bvphq\") pod \"cluster-monitoring-operator-64786d9cbb-8bsph\" (UID: \"b76b8c21-85aa-43de-b1f7-4224e372467a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" Jul 08 18:22:13.918477 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.918463 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jj7r\" (UniqueName: \"kubernetes.io/projected/07b57a1f-a5d2-4bfa-81ac-188e4c33816d-kube-api-access-2jj7r\") pod \"insights-operator-5c847dcfc4-qkmvz\" (UID: \"07b57a1f-a5d2-4bfa-81ac-188e4c33816d\") " pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:13.918968 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.918947 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql2qz\" (UniqueName: \"kubernetes.io/projected/8b408e1d-752d-4537-bbfb-145a3520850f-kube-api-access-ql2qz\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:13.935912 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.935887 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:13.967228 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:13.967205 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" Jul 08 18:22:14.011764 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:14.011731 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fefa4c54-c8d3-48ee-97fd-6358949e39cf-config\") pod \"service-ca-operator-5d46d9ffd4-t58np\" (UID: \"fefa4c54-c8d3-48ee-97fd-6358949e39cf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np" Jul 08 18:22:14.011955 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:14.011836 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fefa4c54-c8d3-48ee-97fd-6358949e39cf-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-t58np\" (UID: \"fefa4c54-c8d3-48ee-97fd-6358949e39cf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np" Jul 08 18:22:14.011955 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:14.011921 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vv2l\" (UniqueName: \"kubernetes.io/projected/fefa4c54-c8d3-48ee-97fd-6358949e39cf-kube-api-access-6vv2l\") pod \"service-ca-operator-5d46d9ffd4-t58np\" (UID: \"fefa4c54-c8d3-48ee-97fd-6358949e39cf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np" Jul 08 18:22:14.012358 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:14.012324 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fefa4c54-c8d3-48ee-97fd-6358949e39cf-config\") pod \"service-ca-operator-5d46d9ffd4-t58np\" (UID: \"fefa4c54-c8d3-48ee-97fd-6358949e39cf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np" Jul 08 18:22:14.014498 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:14.014307 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fefa4c54-c8d3-48ee-97fd-6358949e39cf-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-t58np\" (UID: \"fefa4c54-c8d3-48ee-97fd-6358949e39cf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np" Jul 08 18:22:14.020885 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:14.020838 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vv2l\" (UniqueName: \"kubernetes.io/projected/fefa4c54-c8d3-48ee-97fd-6358949e39cf-kube-api-access-6vv2l\") pod \"service-ca-operator-5d46d9ffd4-t58np\" (UID: \"fefa4c54-c8d3-48ee-97fd-6358949e39cf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np" Jul 08 18:22:14.052146 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:14.052119 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np" Jul 08 18:22:14.091298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:14.091266 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-qkmvz"] Jul 08 18:22:14.095011 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:22:14.094962 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b57a1f_a5d2_4bfa_81ac_188e4c33816d.slice/crio-451a697923bf3c124a1789ce47823be3bdc3dc02bbd4fe2585c6e12bbd8354d4 WatchSource:0}: Error finding container 451a697923bf3c124a1789ce47823be3bdc3dc02bbd4fe2585c6e12bbd8354d4: Status 404 returned error can't find the container with id 451a697923bf3c124a1789ce47823be3bdc3dc02bbd4fe2585c6e12bbd8354d4 Jul 08 18:22:14.180692 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:14.180626 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np"] Jul 08 18:22:14.183382 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:22:14.183354 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfefa4c54_c8d3_48ee_97fd_6358949e39cf.slice/crio-816c0ad7ab78b086b1a6c6e810fb5a7512a77d6213a458362e3d53abdb8f5aa2 WatchSource:0}: Error finding container 816c0ad7ab78b086b1a6c6e810fb5a7512a77d6213a458362e3d53abdb8f5aa2: Status 404 returned error can't find the container with id 816c0ad7ab78b086b1a6c6e810fb5a7512a77d6213a458362e3d53abdb8f5aa2 Jul 08 18:22:14.415397 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:14.415362 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:14.415397 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:14.415400 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-8bsph\" (UID: \"b76b8c21-85aa-43de-b1f7-4224e372467a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" Jul 08 18:22:14.415634 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:14.415441 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:14.415634 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:14.415531 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 18:22:14.415634 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:14.415563 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle podName:8b408e1d-752d-4537-bbfb-145a3520850f nodeName:}" failed. No retries permitted until 2026-07-08 18:22:15.415549833 +0000 UTC m=+256.166859285 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle") pod "router-default-76f778dfc5-nqd5b" (UID: "8b408e1d-752d-4537-bbfb-145a3520850f") : configmap references non-existent config key: service-ca.crt Jul 08 18:22:14.415634 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:14.415587 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls podName:b76b8c21-85aa-43de-b1f7-4224e372467a nodeName:}" failed. No retries permitted until 2026-07-08 18:22:15.415574235 +0000 UTC m=+256.166883689 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-8bsph" (UID: "b76b8c21-85aa-43de-b1f7-4224e372467a") : secret "cluster-monitoring-operator-tls" not found Jul 08 18:22:14.415634 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:14.415532 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 18:22:14.415634 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:14.415613 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs podName:8b408e1d-752d-4537-bbfb-145a3520850f nodeName:}" failed. No retries permitted until 2026-07-08 18:22:15.415607182 +0000 UTC m=+256.166916634 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs") pod "router-default-76f778dfc5-nqd5b" (UID: "8b408e1d-752d-4537-bbfb-145a3520850f") : secret "router-metrics-certs-default" not found Jul 08 18:22:14.438489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:14.438410 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" event={"ID":"07b57a1f-a5d2-4bfa-81ac-188e4c33816d","Type":"ContainerStarted","Data":"451a697923bf3c124a1789ce47823be3bdc3dc02bbd4fe2585c6e12bbd8354d4"} Jul 08 18:22:14.439303 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:14.439282 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np" event={"ID":"fefa4c54-c8d3-48ee-97fd-6358949e39cf","Type":"ContainerStarted","Data":"816c0ad7ab78b086b1a6c6e810fb5a7512a77d6213a458362e3d53abdb8f5aa2"} Jul 08 18:22:15.422742 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:15.422703 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:15.423212 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:15.422777 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:15.423212 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:15.422798 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-8bsph\" (UID: \"b76b8c21-85aa-43de-b1f7-4224e372467a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" Jul 08 18:22:15.423212 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:15.422882 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle podName:8b408e1d-752d-4537-bbfb-145a3520850f nodeName:}" failed. No retries permitted until 2026-07-08 18:22:17.422864225 +0000 UTC m=+258.174173689 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle") pod "router-default-76f778dfc5-nqd5b" (UID: "8b408e1d-752d-4537-bbfb-145a3520850f") : configmap references non-existent config key: service-ca.crt Jul 08 18:22:15.423212 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:15.422912 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 18:22:15.423212 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:15.422944 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 18:22:15.423212 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:15.422963 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls podName:b76b8c21-85aa-43de-b1f7-4224e372467a nodeName:}" failed. No retries permitted until 2026-07-08 18:22:17.422950983 +0000 UTC m=+258.174260442 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-8bsph" (UID: "b76b8c21-85aa-43de-b1f7-4224e372467a") : secret "cluster-monitoring-operator-tls" not found Jul 08 18:22:15.423212 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:15.423002 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs podName:8b408e1d-752d-4537-bbfb-145a3520850f nodeName:}" failed. No retries permitted until 2026-07-08 18:22:17.422984971 +0000 UTC m=+258.174294437 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs") pod "router-default-76f778dfc5-nqd5b" (UID: "8b408e1d-752d-4537-bbfb-145a3520850f") : secret "router-metrics-certs-default" not found Jul 08 18:22:16.445959 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:16.445885 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" event={"ID":"07b57a1f-a5d2-4bfa-81ac-188e4c33816d","Type":"ContainerStarted","Data":"e2539cfc657dc63353606ad5785f1c866ab575c54875b84b5f364c82aa951d10"} Jul 08 18:22:16.464507 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:16.464462 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" podStartSLOduration=1.4554402770000001 podStartE2EDuration="3.464449575s" totalImagesPullingTime="2.009009298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:22:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:22:16.464076034 +0000 UTC m=+257.215385507" watchObservedRunningTime="2026-07-08 18:22:16.464449575 +0000 UTC m=+257.215759049" Jul 08 18:22:16.995897 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:16.995834 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xdvmc"] Jul 08 18:22:17.000150 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.000124 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:17.002979 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.002951 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 18:22:17.003269 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.003242 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 18:22:17.003936 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.003913 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-b7xvt\"" Jul 08 18:22:17.016863 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.016828 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xdvmc"] Jul 08 18:22:17.037450 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.037425 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67415b1e-f842-44cf-89dc-ce26a26d56db-data-volume\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:17.037450 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.037462 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xqdc\" (UniqueName: \"kubernetes.io/projected/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-api-access-8xqdc\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:17.037671 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.037507 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:17.037671 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.037631 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67415b1e-f842-44cf-89dc-ce26a26d56db-crio-socket\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:17.037671 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.037669 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:17.040379 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.040351 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:17.105585 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.105545 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:17.138935 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.138891 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67415b1e-f842-44cf-89dc-ce26a26d56db-crio-socket\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:17.139101 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.138939 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:17.139101 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.139037 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67415b1e-f842-44cf-89dc-ce26a26d56db-data-volume\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:17.139101 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.139052 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67415b1e-f842-44cf-89dc-ce26a26d56db-crio-socket\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:17.139101 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.139063 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xqdc\" (UniqueName: \"kubernetes.io/projected/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-api-access-8xqdc\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:17.139306 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:17.139131 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:22:17.139306 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.139143 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:17.139306 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:17.139204 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm podName:67415b1e-f842-44cf-89dc-ce26a26d56db nodeName:}" failed. No retries permitted until 2026-07-08 18:22:17.639187051 +0000 UTC m=+258.390496506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xdvmc" (UID: "67415b1e-f842-44cf-89dc-ce26a26d56db") : configmap "kube-rbac-proxy" not found Jul 08 18:22:17.139306 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:17.139287 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:22:17.139473 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:17.139347 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls podName:67415b1e-f842-44cf-89dc-ce26a26d56db nodeName:}" failed. No retries permitted until 2026-07-08 18:22:17.63932858 +0000 UTC m=+258.390638048 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xdvmc" (UID: "67415b1e-f842-44cf-89dc-ce26a26d56db") : secret "insights-runtime-extractor-tls" not found Jul 08 18:22:17.139473 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.139444 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67415b1e-f842-44cf-89dc-ce26a26d56db-data-volume\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:17.148027 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.147992 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xqdc\" (UniqueName: \"kubernetes.io/projected/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-api-access-8xqdc\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:17.441790 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.441754 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:17.441790 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.441793 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-8bsph\" (UID: \"b76b8c21-85aa-43de-b1f7-4224e372467a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" Jul 08 18:22:17.442077 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.441870 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:17.442077 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:17.441953 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 18:22:17.442077 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:17.441983 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 18:22:17.442077 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:17.442039 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs podName:8b408e1d-752d-4537-bbfb-145a3520850f nodeName:}" failed. No retries permitted until 2026-07-08 18:22:21.442016846 +0000 UTC m=+262.193326331 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs") pod "router-default-76f778dfc5-nqd5b" (UID: "8b408e1d-752d-4537-bbfb-145a3520850f") : secret "router-metrics-certs-default" not found Jul 08 18:22:17.442077 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:17.442059 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls podName:b76b8c21-85aa-43de-b1f7-4224e372467a nodeName:}" failed. No retries permitted until 2026-07-08 18:22:21.44204936 +0000 UTC m=+262.193358817 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-8bsph" (UID: "b76b8c21-85aa-43de-b1f7-4224e372467a") : secret "cluster-monitoring-operator-tls" not found Jul 08 18:22:17.442077 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:17.442073 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle podName:8b408e1d-752d-4537-bbfb-145a3520850f nodeName:}" failed. No retries permitted until 2026-07-08 18:22:21.442065676 +0000 UTC m=+262.193375132 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle") pod "router-default-76f778dfc5-nqd5b" (UID: "8b408e1d-752d-4537-bbfb-145a3520850f") : configmap references non-existent config key: service-ca.crt Jul 08 18:22:17.450895 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.450864 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np" event={"ID":"fefa4c54-c8d3-48ee-97fd-6358949e39cf","Type":"ContainerStarted","Data":"511b5988d2d973125880169f8f0ae90cab888918aef9d288b0ded4e92858a1d8"} Jul 08 18:22:17.467522 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.467455 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np" podStartSLOduration=1.441941492 podStartE2EDuration="4.467440863s" totalImagesPullingTime="3.025499371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:22:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:22:17.466524229 +0000 UTC m=+258.217833702" watchObservedRunningTime="2026-07-08 18:22:17.467440863 +0000 UTC m=+258.218750337" Jul 08 18:22:17.570169 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.570117 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:17.624399 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.624326 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:17.644568 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.644544 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:17.644654 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:17.644639 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:17.644707 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:17.644676 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:22:17.644744 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:17.644719 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:22:17.644744 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:17.644729 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls podName:67415b1e-f842-44cf-89dc-ce26a26d56db nodeName:}" failed. No retries permitted until 2026-07-08 18:22:18.644715005 +0000 UTC m=+259.396024457 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xdvmc" (UID: "67415b1e-f842-44cf-89dc-ce26a26d56db") : secret "insights-runtime-extractor-tls" not found Jul 08 18:22:17.644823 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:17.644767 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm podName:67415b1e-f842-44cf-89dc-ce26a26d56db nodeName:}" failed. No retries permitted until 2026-07-08 18:22:18.644752209 +0000 UTC m=+259.396061663 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xdvmc" (UID: "67415b1e-f842-44cf-89dc-ce26a26d56db") : configmap "kube-rbac-proxy" not found Jul 08 18:22:18.229344 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:18.229315 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:18.287053 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:18.287026 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:18.651781 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:18.651745 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:18.652206 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:18.651829 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:18.652206 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:18.651906 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:22:18.652206 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:18.651944 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:22:18.652206 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:18.651972 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls podName:67415b1e-f842-44cf-89dc-ce26a26d56db nodeName:}" failed. No retries permitted until 2026-07-08 18:22:20.651958355 +0000 UTC m=+261.403267809 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xdvmc" (UID: "67415b1e-f842-44cf-89dc-ce26a26d56db") : secret "insights-runtime-extractor-tls" not found Jul 08 18:22:18.652206 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:18.651992 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm podName:67415b1e-f842-44cf-89dc-ce26a26d56db nodeName:}" failed. No retries permitted until 2026-07-08 18:22:20.651980019 +0000 UTC m=+261.403289471 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xdvmc" (UID: "67415b1e-f842-44cf-89dc-ce26a26d56db") : configmap "kube-rbac-proxy" not found Jul 08 18:22:18.657084 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:18.657061 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:18.755623 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:18.755577 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:18.930723 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:18.930642 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:19.031568 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:19.031526 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:19.280932 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:19.280864 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rxdnb_dfd1b4d7-5544-40c0-af2a-f22db9888c91/dns-node-resolver/0.log" Jul 08 18:22:19.919224 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:19.919197 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:19.974967 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:19.974937 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:20.080659 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:20.080635 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-smjlq_ec2217c0-db7c-4660-a6f3-990b90cbf03d/node-ca/0.log" Jul 08 18:22:20.566529 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:20.566496 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:20.621351 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:20.621319 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:20.667660 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:20.667615 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:20.667772 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:20.667745 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:22:20.667772 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:20.667747 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:20.667838 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:20.667809 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm podName:67415b1e-f842-44cf-89dc-ce26a26d56db nodeName:}" failed. No retries permitted until 2026-07-08 18:22:24.667790216 +0000 UTC m=+265.419099685 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xdvmc" (UID: "67415b1e-f842-44cf-89dc-ce26a26d56db") : configmap "kube-rbac-proxy" not found Jul 08 18:22:20.667907 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:20.667885 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:22:20.667944 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:20.667934 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls podName:67415b1e-f842-44cf-89dc-ce26a26d56db nodeName:}" failed. No retries permitted until 2026-07-08 18:22:24.667919153 +0000 UTC m=+265.419228608 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xdvmc" (UID: "67415b1e-f842-44cf-89dc-ce26a26d56db") : secret "insights-runtime-extractor-tls" not found Jul 08 18:22:20.879796 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:20.879716 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9fl2r_58fbd0df-e933-4bee-9916-4b786b0205a8/serve-healthcheck-canary/0.log" Jul 08 18:22:21.474886 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:21.474834 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:21.474886 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:21.474885 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-8bsph\" (UID: \"b76b8c21-85aa-43de-b1f7-4224e372467a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" Jul 08 18:22:21.475372 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:21.474987 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 18:22:21.475372 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:21.474991 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 18:22:21.475372 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:21.475041 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:21.475372 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:21.475058 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs podName:8b408e1d-752d-4537-bbfb-145a3520850f nodeName:}" failed. No retries permitted until 2026-07-08 18:22:29.475034401 +0000 UTC m=+270.226343858 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs") pod "router-default-76f778dfc5-nqd5b" (UID: "8b408e1d-752d-4537-bbfb-145a3520850f") : secret "router-metrics-certs-default" not found Jul 08 18:22:21.475372 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:21.475081 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls podName:b76b8c21-85aa-43de-b1f7-4224e372467a nodeName:}" failed. No retries permitted until 2026-07-08 18:22:29.475071717 +0000 UTC m=+270.226381169 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-8bsph" (UID: "b76b8c21-85aa-43de-b1f7-4224e372467a") : secret "cluster-monitoring-operator-tls" not found Jul 08 18:22:21.475372 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:21.475114 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle podName:8b408e1d-752d-4537-bbfb-145a3520850f nodeName:}" failed. No retries permitted until 2026-07-08 18:22:29.47510206 +0000 UTC m=+270.226411511 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle") pod "router-default-76f778dfc5-nqd5b" (UID: "8b408e1d-752d-4537-bbfb-145a3520850f") : configmap references non-existent config key: service-ca.crt Jul 08 18:22:21.492727 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:21.492701 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:24.258222 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:24.258166 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:24.400901 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:24.400837 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:24.629466 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:24.629431 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:24.701326 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:24.701295 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:24.701502 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:24.701368 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:24.701502 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:24.701430 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:22:24.701502 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:24.701449 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:22:24.701502 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:24.701505 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls podName:67415b1e-f842-44cf-89dc-ce26a26d56db nodeName:}" failed. No retries permitted until 2026-07-08 18:22:32.701484068 +0000 UTC m=+273.452793520 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xdvmc" (UID: "67415b1e-f842-44cf-89dc-ce26a26d56db") : secret "insights-runtime-extractor-tls" not found Jul 08 18:22:24.701649 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:24.701529 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm podName:67415b1e-f842-44cf-89dc-ce26a26d56db nodeName:}" failed. No retries permitted until 2026-07-08 18:22:32.701518579 +0000 UTC m=+273.452828034 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xdvmc" (UID: "67415b1e-f842-44cf-89dc-ce26a26d56db") : configmap "kube-rbac-proxy" not found Jul 08 18:22:24.730585 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:24.730553 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:24.831099 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:24.831068 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:24.930745 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:24.930663 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:25.034015 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:25.033983 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:25.132124 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:25.132086 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:25.636902 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:25.636863 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:26.015077 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:26.015003 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:26.612570 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:26.612541 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:27.016648 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:27.016576 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:27.219503 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:27.219467 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:27.618332 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:27.618301 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:27.718765 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:27.718735 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:28.147313 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:28.147280 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:28.249021 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:28.248965 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:28.349020 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:28.348986 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:28.622617 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:28.622587 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:28.820069 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:28.820038 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:29.215888 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:29.215839 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:29.540645 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:29.540556 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:29.540645 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:29.540593 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-8bsph\" (UID: \"b76b8c21-85aa-43de-b1f7-4224e372467a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" Jul 08 18:22:29.540897 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:29.540665 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:29.540897 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:29.540757 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 18:22:29.540897 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:29.540833 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle podName:8b408e1d-752d-4537-bbfb-145a3520850f nodeName:}" failed. No retries permitted until 2026-07-08 18:22:45.54081475 +0000 UTC m=+286.292124202 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle") pod "router-default-76f778dfc5-nqd5b" (UID: "8b408e1d-752d-4537-bbfb-145a3520850f") : configmap references non-existent config key: service-ca.crt Jul 08 18:22:29.540897 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:29.540878 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls podName:b76b8c21-85aa-43de-b1f7-4224e372467a nodeName:}" failed. No retries permitted until 2026-07-08 18:22:45.540866735 +0000 UTC m=+286.292176202 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-8bsph" (UID: "b76b8c21-85aa-43de-b1f7-4224e372467a") : secret "cluster-monitoring-operator-tls" not found Jul 08 18:22:29.543044 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:29.543028 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b408e1d-752d-4537-bbfb-145a3520850f-metrics-certs\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:29.819258 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:29.819228 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:30.218620 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:30.218538 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:30.277118 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:30.277085 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:30.454439 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:30.454408 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:30.618187 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:30.618158 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:30.813504 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:30.813473 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:31.216428 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:31.216395 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:31.629428 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:31.629395 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:32.218487 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:32.218456 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:32.620056 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:32.620031 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:32.767594 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:32.767557 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:32.767980 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:32.767640 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:32.767980 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:32.767749 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:22:32.767980 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:32.767804 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls podName:67415b1e-f842-44cf-89dc-ce26a26d56db nodeName:}" failed. No retries permitted until 2026-07-08 18:22:48.767791627 +0000 UTC m=+289.519101079 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xdvmc" (UID: "67415b1e-f842-44cf-89dc-ce26a26d56db") : secret "insights-runtime-extractor-tls" not found Jul 08 18:22:32.768115 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:32.768082 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67415b1e-f842-44cf-89dc-ce26a26d56db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:33.039972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:33.039890 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:33.219568 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:33.219535 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:33.621261 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:33.621232 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:34.216320 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:34.216294 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:34.617116 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:34.617083 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:34.818820 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:34.818786 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:35.015522 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:35.015447 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:35.385013 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:35.384979 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:35.446596 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:35.446568 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:40.519112 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.519077 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75cf5b4d8b-6v75j"] Jul 08 18:22:40.519496 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:40.519293 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" podUID="8fa3a586-75b0-44a3-a133-9d4caa863f22" Jul 08 18:22:40.593302 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.593271 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56896c69c6-2fsq8"] Jul 08 18:22:40.596335 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.596318 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.616415 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.616389 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56896c69c6-2fsq8"] Jul 08 18:22:40.647960 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.647932 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:40.733542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.733510 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-installation-pull-secrets\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.733542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.733548 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-registry-tls\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.733753 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.733571 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-registry-certificates\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.733753 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.733616 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-bound-sa-token\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.733753 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.733715 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-image-registry-private-configuration\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.733896 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.733779 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-ca-trust-extracted\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.733896 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.733798 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-trusted-ca\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.733896 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.733822 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnn26\" (UniqueName: \"kubernetes.io/projected/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-kube-api-access-rnn26\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.747794 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.747771 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:40.834586 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.834560 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-image-registry-private-configuration\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.834721 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.834609 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-ca-trust-extracted\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.834721 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.834625 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-trusted-ca\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.834721 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.834652 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnn26\" (UniqueName: \"kubernetes.io/projected/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-kube-api-access-rnn26\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.834721 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.834702 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-installation-pull-secrets\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.834946 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.834736 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-registry-tls\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.834946 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.834770 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-registry-certificates\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.834946 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.834798 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-bound-sa-token\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.835084 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.835051 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-ca-trust-extracted\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.835626 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.835605 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-registry-certificates\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.836006 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.835982 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-trusted-ca\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.837216 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.837188 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-image-registry-private-configuration\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.837216 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.837202 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-registry-tls\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.837320 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.837285 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-installation-pull-secrets\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.844445 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.844422 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-bound-sa-token\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.845302 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.845282 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnn26\" (UniqueName: \"kubernetes.io/projected/e46fbbf2-24a7-447c-8904-7e0d7f566bdd-kube-api-access-rnn26\") pod \"image-registry-56896c69c6-2fsq8\" (UID: \"e46fbbf2-24a7-447c-8904-7e0d7f566bdd\") " pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:40.907271 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.907248 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hm9k9\"" Jul 08 18:22:40.915282 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:40.915262 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:41.045584 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.045560 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56896c69c6-2fsq8"] Jul 08 18:22:41.047450 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:22:41.047425 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode46fbbf2_24a7_447c_8904_7e0d7f566bdd.slice/crio-9be3f5a3878678304e02713dcced3d8c8ab3cccef6e6b54464d2c3f8ab7a32fe WatchSource:0}: Error finding container 9be3f5a3878678304e02713dcced3d8c8ab3cccef6e6b54464d2c3f8ab7a32fe: Status 404 returned error can't find the container with id 9be3f5a3878678304e02713dcced3d8c8ab3cccef6e6b54464d2c3f8ab7a32fe Jul 08 18:22:41.212763 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:41.212727 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-cz6kw" podUID="1e9502fd-4d0e-4186-84b2-76b6db2593b1" Jul 08 18:22:41.523093 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.523014 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cz6kw" Jul 08 18:22:41.523093 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.523066 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:22:41.523093 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.523054 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" event={"ID":"e46fbbf2-24a7-447c-8904-7e0d7f566bdd","Type":"ContainerStarted","Data":"4164fb29703a2570e3e7a61b27edce8bbb2b5a44ae4adf5c7ec5f0ec9461e26d"} Jul 08 18:22:41.523602 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.523330 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:22:41.523602 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.523375 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" event={"ID":"e46fbbf2-24a7-447c-8904-7e0d7f566bdd","Type":"ContainerStarted","Data":"9be3f5a3878678304e02713dcced3d8c8ab3cccef6e6b54464d2c3f8ab7a32fe"} Jul 08 18:22:41.530720 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.530703 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:22:41.568568 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.568519 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" podStartSLOduration=1.568503502 podStartE2EDuration="1.568503502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:22:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:22:41.54513406 +0000 UTC m=+282.296443705" watchObservedRunningTime="2026-07-08 18:22:41.568503502 +0000 UTC m=+282.319812976" Jul 08 18:22:41.643573 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.643550 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-certificates\") pod \"8fa3a586-75b0-44a3-a133-9d4caa863f22\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " Jul 08 18:22:41.643733 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.643580 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8fa3a586-75b0-44a3-a133-9d4caa863f22-installation-pull-secrets\") pod \"8fa3a586-75b0-44a3-a133-9d4caa863f22\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " Jul 08 18:22:41.643733 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.643601 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qc56c\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-kube-api-access-qc56c\") pod \"8fa3a586-75b0-44a3-a133-9d4caa863f22\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " Jul 08 18:22:41.643733 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.643618 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-bound-sa-token\") pod \"8fa3a586-75b0-44a3-a133-9d4caa863f22\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " Jul 08 18:22:41.643733 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.643655 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8fa3a586-75b0-44a3-a133-9d4caa863f22-trusted-ca\") pod \"8fa3a586-75b0-44a3-a133-9d4caa863f22\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " Jul 08 18:22:41.643733 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.643684 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8fa3a586-75b0-44a3-a133-9d4caa863f22-ca-trust-extracted\") pod \"8fa3a586-75b0-44a3-a133-9d4caa863f22\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " Jul 08 18:22:41.643733 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.643712 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8fa3a586-75b0-44a3-a133-9d4caa863f22-image-registry-private-configuration\") pod \"8fa3a586-75b0-44a3-a133-9d4caa863f22\" (UID: \"8fa3a586-75b0-44a3-a133-9d4caa863f22\") " Jul 08 18:22:41.644788 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.644050 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8fa3a586-75b0-44a3-a133-9d4caa863f22" (UID: "8fa3a586-75b0-44a3-a133-9d4caa863f22"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:22:41.644788 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.644172 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fa3a586-75b0-44a3-a133-9d4caa863f22-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8fa3a586-75b0-44a3-a133-9d4caa863f22" (UID: "8fa3a586-75b0-44a3-a133-9d4caa863f22"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:22:41.644788 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.644194 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fa3a586-75b0-44a3-a133-9d4caa863f22-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8fa3a586-75b0-44a3-a133-9d4caa863f22" (UID: "8fa3a586-75b0-44a3-a133-9d4caa863f22"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:22:41.646699 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.646679 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fa3a586-75b0-44a3-a133-9d4caa863f22-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8fa3a586-75b0-44a3-a133-9d4caa863f22" (UID: "8fa3a586-75b0-44a3-a133-9d4caa863f22"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:22:41.646759 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.646696 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-kube-api-access-qc56c" (OuterVolumeSpecName: "kube-api-access-qc56c") pod "8fa3a586-75b0-44a3-a133-9d4caa863f22" (UID: "8fa3a586-75b0-44a3-a133-9d4caa863f22"). InnerVolumeSpecName "kube-api-access-qc56c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:22:41.646759 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.646729 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8fa3a586-75b0-44a3-a133-9d4caa863f22" (UID: "8fa3a586-75b0-44a3-a133-9d4caa863f22"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:22:41.646759 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.646744 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fa3a586-75b0-44a3-a133-9d4caa863f22-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8fa3a586-75b0-44a3-a133-9d4caa863f22" (UID: "8fa3a586-75b0-44a3-a133-9d4caa863f22"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:22:41.744786 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.744761 2574 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8fa3a586-75b0-44a3-a133-9d4caa863f22-ca-trust-extracted\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:22:41.744786 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.744785 2574 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8fa3a586-75b0-44a3-a133-9d4caa863f22-image-registry-private-configuration\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:22:41.744951 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.744795 2574 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-certificates\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:22:41.744951 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.744803 2574 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8fa3a586-75b0-44a3-a133-9d4caa863f22-installation-pull-secrets\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:22:41.744951 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.744812 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qc56c\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-kube-api-access-qc56c\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:22:41.744951 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.744820 2574 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-bound-sa-token\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:22:41.744951 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:41.744829 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8fa3a586-75b0-44a3-a133-9d4caa863f22-trusted-ca\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:22:42.526078 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:42.526043 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75cf5b4d8b-6v75j" Jul 08 18:22:42.562276 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:42.562247 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75cf5b4d8b-6v75j"] Jul 08 18:22:42.569640 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:42.569561 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-75cf5b4d8b-6v75j"] Jul 08 18:22:42.651981 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:42.651955 2574 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8fa3a586-75b0-44a3-a133-9d4caa863f22-registry-tls\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:22:43.768286 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:43.768255 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fa3a586-75b0-44a3-a133-9d4caa863f22" path="/var/lib/kubelet/pods/8fa3a586-75b0-44a3-a133-9d4caa863f22/volumes" Jul 08 18:22:44.668082 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:44.668048 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:22:44.670452 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:44.670431 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e9502fd-4d0e-4186-84b2-76b6db2593b1-metrics-tls\") pod \"dns-default-cz6kw\" (UID: \"1e9502fd-4d0e-4186-84b2-76b6db2593b1\") " pod="openshift-dns/dns-default-cz6kw" Jul 08 18:22:44.826523 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:44.826493 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d4sdr\"" Jul 08 18:22:44.834580 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:44.834561 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cz6kw" Jul 08 18:22:44.957967 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:44.957946 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cz6kw"] Jul 08 18:22:44.960438 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:22:44.960413 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e9502fd_4d0e_4186_84b2_76b6db2593b1.slice/crio-6963ade3599a0063c6e5483c7204729653296c7c9c2ea8ac15e904c167d4f830 WatchSource:0}: Error finding container 6963ade3599a0063c6e5483c7204729653296c7c9c2ea8ac15e904c167d4f830: Status 404 returned error can't find the container with id 6963ade3599a0063c6e5483c7204729653296c7c9c2ea8ac15e904c167d4f830 Jul 08 18:22:45.535514 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:45.535475 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cz6kw" event={"ID":"1e9502fd-4d0e-4186-84b2-76b6db2593b1","Type":"ContainerStarted","Data":"6963ade3599a0063c6e5483c7204729653296c7c9c2ea8ac15e904c167d4f830"} Jul 08 18:22:45.575290 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:45.575259 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-8bsph\" (UID: \"b76b8c21-85aa-43de-b1f7-4224e372467a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" Jul 08 18:22:45.575469 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:45.575322 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:45.576098 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:45.576047 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b408e1d-752d-4537-bbfb-145a3520850f-service-ca-bundle\") pod \"router-default-76f778dfc5-nqd5b\" (UID: \"8b408e1d-752d-4537-bbfb-145a3520850f\") " pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:45.578199 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:45.578174 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76b8c21-85aa-43de-b1f7-4224e372467a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-8bsph\" (UID: \"b76b8c21-85aa-43de-b1f7-4224e372467a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" Jul 08 18:22:45.772862 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:45.772815 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" Jul 08 18:22:45.777775 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:45.777749 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:45.912323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:45.912293 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph"] Jul 08 18:22:45.916072 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:22:45.916015 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76b8c21_85aa_43de_b1f7_4224e372467a.slice/crio-f484a1bc6cdcc6b122e6371158d8b55abf24746179face3506a28139a0a6a2bf WatchSource:0}: Error finding container f484a1bc6cdcc6b122e6371158d8b55abf24746179face3506a28139a0a6a2bf: Status 404 returned error can't find the container with id f484a1bc6cdcc6b122e6371158d8b55abf24746179face3506a28139a0a6a2bf Jul 08 18:22:45.936008 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:45.935981 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-76f778dfc5-nqd5b"] Jul 08 18:22:45.940359 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:22:45.940334 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b408e1d_752d_4537_bbfb_145a3520850f.slice/crio-95f85fed628d151b39484d88829054af1fc7636dd1d1bebde05f673c731de43a WatchSource:0}: Error finding container 95f85fed628d151b39484d88829054af1fc7636dd1d1bebde05f673c731de43a: Status 404 returned error can't find the container with id 95f85fed628d151b39484d88829054af1fc7636dd1d1bebde05f673c731de43a Jul 08 18:22:46.540699 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:46.540664 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-76f778dfc5-nqd5b" event={"ID":"8b408e1d-752d-4537-bbfb-145a3520850f","Type":"ContainerStarted","Data":"0946f004d6cea872b2f947184f0f86c6cdaec636fe1662aa0256e336fd93cc25"} Jul 08 18:22:46.540900 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:46.540718 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-76f778dfc5-nqd5b" event={"ID":"8b408e1d-752d-4537-bbfb-145a3520850f","Type":"ContainerStarted","Data":"95f85fed628d151b39484d88829054af1fc7636dd1d1bebde05f673c731de43a"} Jul 08 18:22:46.541966 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:46.541928 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" event={"ID":"b76b8c21-85aa-43de-b1f7-4224e372467a","Type":"ContainerStarted","Data":"f484a1bc6cdcc6b122e6371158d8b55abf24746179face3506a28139a0a6a2bf"} Jul 08 18:22:46.561024 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:46.560943 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-76f778dfc5-nqd5b" podStartSLOduration=33.56092647 podStartE2EDuration="33.56092647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:22:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:22:46.560685038 +0000 UTC m=+287.311994527" watchObservedRunningTime="2026-07-08 18:22:46.56092647 +0000 UTC m=+287.312235925" Jul 08 18:22:46.778007 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:46.777961 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:46.780390 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:46.780369 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:47.545726 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:47.545695 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:47.547212 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:47.547190 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-76f778dfc5-nqd5b" Jul 08 18:22:48.417446 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:48.417410 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:48.550686 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:48.550649 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cz6kw" event={"ID":"1e9502fd-4d0e-4186-84b2-76b6db2593b1","Type":"ContainerStarted","Data":"8d906ceff341672aafe2fef1f4b8927fc7d3ae2b9f6319c94c4fed0a9108429a"} Jul 08 18:22:48.551131 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:48.550733 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cz6kw" event={"ID":"1e9502fd-4d0e-4186-84b2-76b6db2593b1","Type":"ContainerStarted","Data":"db92ee7fde9b75cbcd0ea7b2c66b963e65f2bc5b184442670a8fdb40aa83d158"} Jul 08 18:22:48.551131 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:48.550764 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cz6kw" Jul 08 18:22:48.552561 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:48.552541 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" event={"ID":"b76b8c21-85aa-43de-b1f7-4224e372467a","Type":"ContainerStarted","Data":"8b10e6a3e543f67ae7e0d5de1c10fb8f0d4534be38ed328d451c795666ab3c7e"} Jul 08 18:22:48.573628 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:48.573580 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cz6kw" podStartSLOduration=251.482348974 podStartE2EDuration="4m14.573566355s" totalImagesPullingTime="3.091217381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:22:48.572219759 +0000 UTC m=+289.323529233" watchObservedRunningTime="2026-07-08 18:22:48.573566355 +0000 UTC m=+289.324875836" Jul 08 18:22:48.589513 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:48.589466 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-8bsph" podStartSLOduration=33.452711664 podStartE2EDuration="35.589449135s" totalImagesPullingTime="2.136737471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:22:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:22:48.588382584 +0000 UTC m=+289.339692061" watchObservedRunningTime="2026-07-08 18:22:48.589449135 +0000 UTC m=+289.340758608" Jul 08 18:22:48.803056 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:48.802971 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:48.805362 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:48.805333 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67415b1e-f842-44cf-89dc-ce26a26d56db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xdvmc\" (UID: \"67415b1e-f842-44cf-89dc-ce26a26d56db\") " pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:48.810371 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:48.810342 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xdvmc" Jul 08 18:22:48.942064 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:48.941881 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xdvmc"] Jul 08 18:22:48.945082 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:22:48.945056 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67415b1e_f842_44cf_89dc_ce26a26d56db.slice/crio-da80ed702b0c91ed75badf5431a68df624bac3618d2697e86d1ad36159141a02 WatchSource:0}: Error finding container da80ed702b0c91ed75badf5431a68df624bac3618d2697e86d1ad36159141a02: Status 404 returned error can't find the container with id da80ed702b0c91ed75badf5431a68df624bac3618d2697e86d1ad36159141a02 Jul 08 18:22:49.560794 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:49.560756 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xdvmc" event={"ID":"67415b1e-f842-44cf-89dc-ce26a26d56db","Type":"ContainerStarted","Data":"da80ed702b0c91ed75badf5431a68df624bac3618d2697e86d1ad36159141a02"} Jul 08 18:22:49.931135 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:49.931051 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:52.040155 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:52.040126 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:52.140804 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:52.140768 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:52.603302 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:52.603262 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:52.677841 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:52.677812 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:52.766749 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:52.766722 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ts8sd" podUID="0d868453-b10b-4dd9-b843-91de3f1b48d3" Jul 08 18:22:52.783300 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:52.783241 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:52.935964 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:52.935873 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:53.037285 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:53.037257 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:53.576156 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:53.576124 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xdvmc" event={"ID":"67415b1e-f842-44cf-89dc-ce26a26d56db","Type":"ContainerStarted","Data":"0554b1e13094efec4b896e778e26320ef0bb6f8110646838f97c8d3ca1f24fea"} Jul 08 18:22:53.576515 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:53.576205 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xdvmc" event={"ID":"67415b1e-f842-44cf-89dc-ce26a26d56db","Type":"ContainerStarted","Data":"f5114f7f6c37f7f462de69c67c07bc13f48d808c2f736ba78ab481bd2d296805"} Jul 08 18:22:55.815047 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:55.815012 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:56.548267 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:56.548234 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:56.587894 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:56.587839 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xdvmc" event={"ID":"67415b1e-f842-44cf-89dc-ce26a26d56db","Type":"ContainerStarted","Data":"4aa74309a8643e6ac1669d8f2503c6ee77664ef0f0ce1e1e496aef7d750dc9e4"} Jul 08 18:22:56.608544 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:56.608499 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xdvmc" podStartSLOduration=33.786002153 podStartE2EDuration="40.608485756s" totalImagesPullingTime="6.822483603s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:22:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:22:56.606714026 +0000 UTC m=+297.358023494" watchObservedRunningTime="2026-07-08 18:22:56.608485756 +0000 UTC m=+297.359795229" Jul 08 18:22:56.614911 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:56.614888 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:57.742639 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:57.742602 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:57.944726 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:57.944681 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:57.994459 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:57.994377 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc"] Jul 08 18:22:57.997645 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:57.997623 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.000670 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.000653 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 08 18:22:58.000818 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.000802 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 18:22:58.001786 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.001767 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 08 18:22:58.001904 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.001790 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qc8s7\"" Jul 08 18:22:58.001904 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.001884 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 08 18:22:58.036900 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.036866 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc"] Jul 08 18:22:58.037574 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.037556 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b2r5c"] Jul 08 18:22:58.040590 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.040576 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.043958 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.043940 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 18:22:58.044151 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.044133 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 18:22:58.044229 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.044193 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hbm8s\"" Jul 08 18:22:58.044333 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.044319 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 18:22:58.070090 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.070048 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:58.165491 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.165461 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:58.185330 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.185300 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a63305b0-9222-492b-8f3d-b3860f225b31-node-exporter-textfile\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.185330 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.185333 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a63305b0-9222-492b-8f3d-b3860f225b31-node-exporter-accelerators-collector-config\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.185542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.185356 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a63305b0-9222-492b-8f3d-b3860f225b31-sys\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.185542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.185379 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a63305b0-9222-492b-8f3d-b3860f225b31-metrics-client-ca\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.185542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.185399 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f64fdac-6907-4dba-bc6d-491cda836b65-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.185542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.185416 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wjd9\" (UniqueName: \"kubernetes.io/projected/3f64fdac-6907-4dba-bc6d-491cda836b65-kube-api-access-8wjd9\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.185542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.185452 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a63305b0-9222-492b-8f3d-b3860f225b31-root\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.185542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.185469 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a63305b0-9222-492b-8f3d-b3860f225b31-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.185542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.185490 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3f64fdac-6907-4dba-bc6d-491cda836b65-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.185542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.185508 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3f64fdac-6907-4dba-bc6d-491cda836b65-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.185542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.185534 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a63305b0-9222-492b-8f3d-b3860f225b31-node-exporter-tls\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.185823 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.185602 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a63305b0-9222-492b-8f3d-b3860f225b31-node-exporter-wtmp\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.185823 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.185633 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f64fdac-6907-4dba-bc6d-491cda836b65-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.185823 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.185654 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f64fdac-6907-4dba-bc6d-491cda836b65-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.185823 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.185680 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbkds\" (UniqueName: \"kubernetes.io/projected/a63305b0-9222-492b-8f3d-b3860f225b31-kube-api-access-zbkds\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.265453 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.265365 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:58.286504 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.286464 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a63305b0-9222-492b-8f3d-b3860f225b31-node-exporter-wtmp\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.286632 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.286514 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f64fdac-6907-4dba-bc6d-491cda836b65-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.286632 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.286534 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f64fdac-6907-4dba-bc6d-491cda836b65-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.286632 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.286557 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbkds\" (UniqueName: \"kubernetes.io/projected/a63305b0-9222-492b-8f3d-b3860f225b31-kube-api-access-zbkds\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.286769 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.286681 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a63305b0-9222-492b-8f3d-b3860f225b31-node-exporter-wtmp\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.286769 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:58.286705 2574 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 08 18:22:58.286876 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:22:58.286782 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f64fdac-6907-4dba-bc6d-491cda836b65-kube-state-metrics-tls podName:3f64fdac-6907-4dba-bc6d-491cda836b65 nodeName:}" failed. No retries permitted until 2026-07-08 18:22:58.786760816 +0000 UTC m=+299.538070276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3f64fdac-6907-4dba-bc6d-491cda836b65-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-lrbbc" (UID: "3f64fdac-6907-4dba-bc6d-491cda836b65") : secret "kube-state-metrics-tls" not found Jul 08 18:22:58.286876 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.286801 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a63305b0-9222-492b-8f3d-b3860f225b31-node-exporter-textfile\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.286876 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.286827 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a63305b0-9222-492b-8f3d-b3860f225b31-node-exporter-accelerators-collector-config\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.287038 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.286884 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a63305b0-9222-492b-8f3d-b3860f225b31-sys\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.287038 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.286913 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a63305b0-9222-492b-8f3d-b3860f225b31-metrics-client-ca\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.287038 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.286943 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f64fdac-6907-4dba-bc6d-491cda836b65-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.287038 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.286970 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wjd9\" (UniqueName: \"kubernetes.io/projected/3f64fdac-6907-4dba-bc6d-491cda836b65-kube-api-access-8wjd9\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.287038 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.286992 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a63305b0-9222-492b-8f3d-b3860f225b31-sys\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.287038 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.287009 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a63305b0-9222-492b-8f3d-b3860f225b31-root\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.287038 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.287033 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a63305b0-9222-492b-8f3d-b3860f225b31-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.287369 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.287076 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3f64fdac-6907-4dba-bc6d-491cda836b65-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.287369 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.287107 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3f64fdac-6907-4dba-bc6d-491cda836b65-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.287369 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.287135 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a63305b0-9222-492b-8f3d-b3860f225b31-node-exporter-tls\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.287369 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.287233 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a63305b0-9222-492b-8f3d-b3860f225b31-node-exporter-textfile\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.287369 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.287300 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a63305b0-9222-492b-8f3d-b3860f225b31-root\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.287619 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.287595 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a63305b0-9222-492b-8f3d-b3860f225b31-node-exporter-accelerators-collector-config\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.287794 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.287694 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3f64fdac-6907-4dba-bc6d-491cda836b65-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.287903 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.287794 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a63305b0-9222-492b-8f3d-b3860f225b31-metrics-client-ca\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.287903 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.287810 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f64fdac-6907-4dba-bc6d-491cda836b65-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.287982 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.287954 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3f64fdac-6907-4dba-bc6d-491cda836b65-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.289178 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.289158 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f64fdac-6907-4dba-bc6d-491cda836b65-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.290277 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.290262 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a63305b0-9222-492b-8f3d-b3860f225b31-node-exporter-tls\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.290383 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.290364 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a63305b0-9222-492b-8f3d-b3860f225b31-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.295068 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.295043 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbkds\" (UniqueName: \"kubernetes.io/projected/a63305b0-9222-492b-8f3d-b3860f225b31-kube-api-access-zbkds\") pod \"node-exporter-b2r5c\" (UID: \"a63305b0-9222-492b-8f3d-b3860f225b31\") " pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.295565 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.295543 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wjd9\" (UniqueName: \"kubernetes.io/projected/3f64fdac-6907-4dba-bc6d-491cda836b65-kube-api-access-8wjd9\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.349365 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.349338 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b2r5c" Jul 08 18:22:58.361080 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:22:58.361051 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda63305b0_9222_492b_8f3d_b3860f225b31.slice/crio-cd0b6758d8f679b3200ed86ac3ebfc1e02d6be3d51d7afbb0b4afd5af75ec0f5 WatchSource:0}: Error finding container cd0b6758d8f679b3200ed86ac3ebfc1e02d6be3d51d7afbb0b4afd5af75ec0f5: Status 404 returned error can't find the container with id cd0b6758d8f679b3200ed86ac3ebfc1e02d6be3d51d7afbb0b4afd5af75ec0f5 Jul 08 18:22:58.365929 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.365898 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:58.467901 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.467875 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:58.562956 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.562933 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cz6kw" Jul 08 18:22:58.566911 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.566887 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:58.598466 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.598429 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b2r5c" event={"ID":"a63305b0-9222-492b-8f3d-b3860f225b31","Type":"ContainerStarted","Data":"cd0b6758d8f679b3200ed86ac3ebfc1e02d6be3d51d7afbb0b4afd5af75ec0f5"} Jul 08 18:22:58.667689 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.667662 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:58.767553 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.767523 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:58.790827 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.790795 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f64fdac-6907-4dba-bc6d-491cda836b65-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.793416 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.793384 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f64fdac-6907-4dba-bc6d-491cda836b65-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-lrbbc\" (UID: \"3f64fdac-6907-4dba-bc6d-491cda836b65\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.906407 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.906338 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" Jul 08 18:22:58.983458 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:58.983430 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:59.041881 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:59.041833 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc"] Jul 08 18:22:59.045319 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:22:59.045288 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f64fdac_6907_4dba_bc6d_491cda836b65.slice/crio-964cf621ffa89861e7b91a722e6f5c44c0df7e6c41d66c2c8b7f1ee933c4c789 WatchSource:0}: Error finding container 964cf621ffa89861e7b91a722e6f5c44c0df7e6c41d66c2c8b7f1ee933c4c789: Status 404 returned error can't find the container with id 964cf621ffa89861e7b91a722e6f5c44c0df7e6c41d66c2c8b7f1ee933c4c789 Jul 08 18:22:59.051374 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:59.051352 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:59.503351 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:59.503317 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:59.610774 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:59.610630 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" event={"ID":"3f64fdac-6907-4dba-bc6d-491cda836b65","Type":"ContainerStarted","Data":"964cf621ffa89861e7b91a722e6f5c44c0df7e6c41d66c2c8b7f1ee933c4c789"} Jul 08 18:22:59.626464 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:22:59.626440 2574 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 18:23:00.616761 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:00.616724 2574 generic.go:356] "Generic (PLEG): container finished" podID="a63305b0-9222-492b-8f3d-b3860f225b31" containerID="245781adc6f827b0392331dac90daacfe3668a9b036ac280a39daf7cc5b6d5d9" exitCode=0 Jul 08 18:23:00.619469 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:00.616787 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b2r5c" event={"ID":"a63305b0-9222-492b-8f3d-b3860f225b31","Type":"ContainerDied","Data":"245781adc6f827b0392331dac90daacfe3668a9b036ac280a39daf7cc5b6d5d9"} Jul 08 18:23:00.930062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:00.929972 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:01.010791 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.010758 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7d868f677b-hlglt"] Jul 08 18:23:01.016728 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.016701 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.019341 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.019321 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 08 18:23:01.019750 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.019468 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 08 18:23:01.019750 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.019496 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 08 18:23:01.019750 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.019542 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 08 18:23:01.019750 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.019439 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 08 18:23:01.019750 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.019497 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-5jdlf\"" Jul 08 18:23:01.019750 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.019389 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-kdt4e665lfue\"" Jul 08 18:23:01.035424 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.035294 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d868f677b-hlglt"] Jul 08 18:23:01.041502 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.041439 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:01.110070 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.110036 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.110070 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.110074 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-grpc-tls\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.110287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.110136 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx25l\" (UniqueName: \"kubernetes.io/projected/990d6988-42df-4af3-bec7-4b652a019f99-kube-api-access-sx25l\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.110287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.110192 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.110287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.110213 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-thanos-querier-tls\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.110287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.110231 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.110287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.110266 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/990d6988-42df-4af3-bec7-4b652a019f99-metrics-client-ca\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.110525 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.110300 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.210978 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.210881 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.210978 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.210923 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-grpc-tls\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.211216 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.210990 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx25l\" (UniqueName: \"kubernetes.io/projected/990d6988-42df-4af3-bec7-4b652a019f99-kube-api-access-sx25l\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.211216 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.211048 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.211216 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.211075 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-thanos-querier-tls\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.211216 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.211100 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.211216 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.211146 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/990d6988-42df-4af3-bec7-4b652a019f99-metrics-client-ca\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.211216 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.211178 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.212180 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.212129 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/990d6988-42df-4af3-bec7-4b652a019f99-metrics-client-ca\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.213834 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.213804 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.214193 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.214158 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.214304 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.214289 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.214489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.214467 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-grpc-tls\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.214626 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.214575 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-thanos-querier-tls\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.215090 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.214995 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/990d6988-42df-4af3-bec7-4b652a019f99-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.219532 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.219513 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx25l\" (UniqueName: \"kubernetes.io/projected/990d6988-42df-4af3-bec7-4b652a019f99-kube-api-access-sx25l\") pod \"thanos-querier-7d868f677b-hlglt\" (UID: \"990d6988-42df-4af3-bec7-4b652a019f99\") " pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.242990 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.242961 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:01.338957 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.338929 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:01.340822 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.340796 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:01.470324 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.470253 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d868f677b-hlglt"] Jul 08 18:23:01.473191 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:23:01.473162 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod990d6988_42df_4af3_bec7_4b652a019f99.slice/crio-a45ee2fa713aff970b63120a0acf10fe9a2834b757999f468510952a5761d67c WatchSource:0}: Error finding container a45ee2fa713aff970b63120a0acf10fe9a2834b757999f468510952a5761d67c: Status 404 returned error can't find the container with id a45ee2fa713aff970b63120a0acf10fe9a2834b757999f468510952a5761d67c Jul 08 18:23:01.475257 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.475235 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:23:01.621427 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.621393 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" event={"ID":"990d6988-42df-4af3-bec7-4b652a019f99","Type":"ContainerStarted","Data":"a45ee2fa713aff970b63120a0acf10fe9a2834b757999f468510952a5761d67c"} Jul 08 18:23:01.623440 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.623411 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b2r5c" event={"ID":"a63305b0-9222-492b-8f3d-b3860f225b31","Type":"ContainerStarted","Data":"c108d017ac842f33f44bbcbc338b430da8cc727219697a94613f1187a2433fa6"} Jul 08 18:23:01.623582 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.623466 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b2r5c" event={"ID":"a63305b0-9222-492b-8f3d-b3860f225b31","Type":"ContainerStarted","Data":"128a854d744a9b6c4b88625ac5d8699c9339d26e307fd6acab71095c1eeaeccc"} Jul 08 18:23:01.625443 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.625419 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" event={"ID":"3f64fdac-6907-4dba-bc6d-491cda836b65","Type":"ContainerStarted","Data":"bb0b4d0f9883576e73ba8934e5fc464afe056d0147a8f7e50dbf8419afe15128"} Jul 08 18:23:01.625540 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.625461 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" event={"ID":"3f64fdac-6907-4dba-bc6d-491cda836b65","Type":"ContainerStarted","Data":"956d1bf7fdfd5d71e2480d3c793da11a04e70a893791875dc894d64d5beb3268"} Jul 08 18:23:01.625540 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.625482 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" event={"ID":"3f64fdac-6907-4dba-bc6d-491cda836b65","Type":"ContainerStarted","Data":"4a6b5e6ed61689db9129e377f5d99810d50656adeb460a40544dd56a8d040bcc"} Jul 08 18:23:01.645554 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.645509 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b2r5c" podStartSLOduration=3.35287158 podStartE2EDuration="4.645492706s" totalImagesPullingTime="1.220626379s" totalInitContainerRuntime="71.994747ms" isStatefulPod=true podCreationTimestamp="2026-07-08 18:22:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:23:01.644445421 +0000 UTC m=+302.395754897" watchObservedRunningTime="2026-07-08 18:23:01.645492706 +0000 UTC m=+302.396802211" Jul 08 18:23:01.661541 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:01.661491 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrbbc" podStartSLOduration=2.876626421 podStartE2EDuration="4.661478339s" totalImagesPullingTime="1.784851918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:22:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:23:01.66044051 +0000 UTC m=+302.411749984" watchObservedRunningTime="2026-07-08 18:23:01.661478339 +0000 UTC m=+302.412787812" Jul 08 18:23:02.531059 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:02.531030 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56896c69c6-2fsq8" Jul 08 18:23:02.689340 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:02.689306 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:02.758607 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:02.758573 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:02.887857 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:02.887821 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:02.988091 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:02.988059 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:03.243654 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.243575 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-859bf86486-s6rv4"] Jul 08 18:23:03.246962 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.246943 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.249884 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.249838 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 18:23:03.249884 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.249870 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 18:23:03.250028 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.249897 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 18:23:03.250104 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.250089 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 18:23:03.250184 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.250170 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 18:23:03.250267 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.250251 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-fl5km\"" Jul 08 18:23:03.254966 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.254946 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 18:23:03.268045 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.268020 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-859bf86486-s6rv4"] Jul 08 18:23:03.330581 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.330543 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skm4n\" (UniqueName: \"kubernetes.io/projected/1f231c29-f57a-48b2-b71e-c5c2f80be098-kube-api-access-skm4n\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.330745 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.330604 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1f231c29-f57a-48b2-b71e-c5c2f80be098-telemeter-client-tls\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.330745 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.330637 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f231c29-f57a-48b2-b71e-c5c2f80be098-telemeter-trusted-ca-bundle\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.330745 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.330679 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f231c29-f57a-48b2-b71e-c5c2f80be098-metrics-client-ca\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.330930 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.330739 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f231c29-f57a-48b2-b71e-c5c2f80be098-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.330930 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.330787 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1f231c29-f57a-48b2-b71e-c5c2f80be098-federate-client-tls\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.330930 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.330818 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f231c29-f57a-48b2-b71e-c5c2f80be098-serving-certs-ca-bundle\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.331061 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.330951 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1f231c29-f57a-48b2-b71e-c5c2f80be098-secret-telemeter-client\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.426562 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.426528 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:03.431589 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.431561 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f231c29-f57a-48b2-b71e-c5c2f80be098-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.431711 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.431604 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1f231c29-f57a-48b2-b71e-c5c2f80be098-federate-client-tls\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.431711 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.431630 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f231c29-f57a-48b2-b71e-c5c2f80be098-serving-certs-ca-bundle\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.431711 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.431699 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1f231c29-f57a-48b2-b71e-c5c2f80be098-secret-telemeter-client\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.431889 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.431741 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skm4n\" (UniqueName: \"kubernetes.io/projected/1f231c29-f57a-48b2-b71e-c5c2f80be098-kube-api-access-skm4n\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.431889 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.431783 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1f231c29-f57a-48b2-b71e-c5c2f80be098-telemeter-client-tls\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.431889 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.431818 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f231c29-f57a-48b2-b71e-c5c2f80be098-telemeter-trusted-ca-bundle\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.432049 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.431917 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f231c29-f57a-48b2-b71e-c5c2f80be098-metrics-client-ca\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.432742 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.432528 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f231c29-f57a-48b2-b71e-c5c2f80be098-serving-certs-ca-bundle\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.432742 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.432700 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f231c29-f57a-48b2-b71e-c5c2f80be098-metrics-client-ca\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.432742 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.432707 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f231c29-f57a-48b2-b71e-c5c2f80be098-telemeter-trusted-ca-bundle\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.434279 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.434261 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f231c29-f57a-48b2-b71e-c5c2f80be098-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.434633 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.434605 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1f231c29-f57a-48b2-b71e-c5c2f80be098-telemeter-client-tls\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.434752 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.434738 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1f231c29-f57a-48b2-b71e-c5c2f80be098-federate-client-tls\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.434869 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.434829 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1f231c29-f57a-48b2-b71e-c5c2f80be098-secret-telemeter-client\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.440878 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.440831 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skm4n\" (UniqueName: \"kubernetes.io/projected/1f231c29-f57a-48b2-b71e-c5c2f80be098-kube-api-access-skm4n\") pod \"telemeter-client-859bf86486-s6rv4\" (UID: \"1f231c29-f57a-48b2-b71e-c5c2f80be098\") " pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.556737 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.556657 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" Jul 08 18:23:03.701463 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:03.701421 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-859bf86486-s6rv4"] Jul 08 18:23:04.199267 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.199236 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:04.203149 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.203112 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:23:04.208356 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.208335 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.211054 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.210957 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 18:23:04.211183 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.211057 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 18:23:04.211183 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.211061 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 18:23:04.211183 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.211070 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xnqfj\"" Jul 08 18:23:04.211183 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.211094 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 18:23:04.211397 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.211206 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 18:23:04.211526 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.211511 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 18:23:04.211974 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.211949 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 18:23:04.212381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.212150 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 18:23:04.212381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.212193 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 18:23:04.212381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.212211 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 18:23:04.212381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.212235 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 18:23:04.212381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.212270 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 18:23:04.212381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.212213 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 18:23:04.212715 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.212543 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 18:23:04.213715 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.213698 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9bt64jlnnr16n\"" Jul 08 18:23:04.215755 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.215732 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 18:23:04.227127 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:23:04.227103 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f231c29_f57a_48b2_b71e_c5c2f80be098.slice/crio-23f6b51fb34a23116d081c85efad78c615fb5a9d637305428adb6a33b4d27a83 WatchSource:0}: Error finding container 23f6b51fb34a23116d081c85efad78c615fb5a9d637305428adb6a33b4d27a83: Status 404 returned error can't find the container with id 23f6b51fb34a23116d081c85efad78c615fb5a9d637305428adb6a33b4d27a83 Jul 08 18:23:04.227264 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.227250 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:23:04.240001 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.239978 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240073 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240024 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f4pv\" (UniqueName: \"kubernetes.io/projected/08b800d1-45ed-460a-b934-6f76d00d2f6d-kube-api-access-6f4pv\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240073 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240043 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240073 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240060 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240186 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240084 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240186 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240117 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240186 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240138 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240186 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240165 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240320 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240192 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240320 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240244 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08b800d1-45ed-460a-b934-6f76d00d2f6d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240320 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240264 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240320 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240282 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08b800d1-45ed-460a-b934-6f76d00d2f6d-config-out\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240445 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240317 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240445 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240339 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240445 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240356 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240445 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240371 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-config\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240445 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240417 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240601 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240449 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240601 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240474 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-web-config\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.240601 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.240495 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341093 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341071 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f4pv\" (UniqueName: \"kubernetes.io/projected/08b800d1-45ed-460a-b934-6f76d00d2f6d-kube-api-access-6f4pv\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341187 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341113 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341187 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341145 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341187 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341168 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341390 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341208 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341390 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341233 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341390 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341263 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341390 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341293 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341390 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341327 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08b800d1-45ed-460a-b934-6f76d00d2f6d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341390 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341355 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341697 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341395 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08b800d1-45ed-460a-b934-6f76d00d2f6d-config-out\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341697 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341439 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341697 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341466 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341697 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341491 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341697 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341518 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-config\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341697 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341555 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341697 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341579 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341697 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341601 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-web-config\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341697 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341623 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.341697 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341667 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.342388 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.341760 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.342480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.342454 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.343288 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:23:04.343089 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-trusted-ca-bundle podName:08b800d1-45ed-460a-b934-6f76d00d2f6d nodeName:}" failed. No retries permitted until 2026-07-08 18:23:04.843070188 +0000 UTC m=+305.594379649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-trusted-ca-bundle") pod "prometheus-k8s-0" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d") : configmap references non-existent config key: ca-bundle.crt Jul 08 18:23:04.343288 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.343276 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.345480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.345450 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.345941 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.345573 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.346282 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.346201 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.346282 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.346213 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.346282 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.346215 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.347369 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.347348 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.347510 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.347403 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-web-config\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.347766 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.347744 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-config\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.347998 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.347977 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08b800d1-45ed-460a-b934-6f76d00d2f6d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.348090 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.348069 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.348184 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.348164 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.348470 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.348449 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.348570 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.348547 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.348643 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.348628 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.349123 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.349103 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f4pv\" (UniqueName: \"kubernetes.io/projected/08b800d1-45ed-460a-b934-6f76d00d2f6d-kube-api-access-6f4pv\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.349200 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.349140 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08b800d1-45ed-460a-b934-6f76d00d2f6d-config-out\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.645388 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.645354 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" event={"ID":"990d6988-42df-4af3-bec7-4b652a019f99","Type":"ContainerStarted","Data":"9e2a145072a0fa42684fc0c1a3887f0673683dfe031b9560aa0d433cba3688da"} Jul 08 18:23:04.645558 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.645457 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" event={"ID":"990d6988-42df-4af3-bec7-4b652a019f99","Type":"ContainerStarted","Data":"52c5e9ce4544d2468fd787435cda0bb4fb6026f00473b3976989a5dce7d4c975"} Jul 08 18:23:04.645558 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.645489 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" event={"ID":"990d6988-42df-4af3-bec7-4b652a019f99","Type":"ContainerStarted","Data":"ac25801b0de1345a87f04546bb6fa508e66bfd248b740def8fc6b7f308129d42"} Jul 08 18:23:04.646372 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.646350 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" event={"ID":"1f231c29-f57a-48b2-b71e-c5c2f80be098","Type":"ContainerStarted","Data":"23f6b51fb34a23116d081c85efad78c615fb5a9d637305428adb6a33b4d27a83"} Jul 08 18:23:04.846087 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.846037 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:04.846766 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:04.846744 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:05.121464 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:05.121369 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:05.280712 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:05.280677 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:23:05.286361 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:23:05.284770 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08b800d1_45ed_460a_b934_6f76d00d2f6d.slice/crio-d2deb4d516526f136c8607821244d1e83d77a83daa5ff42fa4cc816cdb022ca3 WatchSource:0}: Error finding container d2deb4d516526f136c8607821244d1e83d77a83daa5ff42fa4cc816cdb022ca3: Status 404 returned error can't find the container with id d2deb4d516526f136c8607821244d1e83d77a83daa5ff42fa4cc816cdb022ca3 Jul 08 18:23:05.384181 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:05.384080 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:05.483172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:05.483107 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:05.628155 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:05.628126 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:05.652362 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:05.652334 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" event={"ID":"990d6988-42df-4af3-bec7-4b652a019f99","Type":"ContainerStarted","Data":"fc054e22cf71a7c89692823bbfa34e250eaa2b489c0870e14b67199f5bc9e7b7"} Jul 08 18:23:05.653595 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:05.653573 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"08b800d1-45ed-460a-b934-6f76d00d2f6d","Type":"ContainerStarted","Data":"d2deb4d516526f136c8607821244d1e83d77a83daa5ff42fa4cc816cdb022ca3"} Jul 08 18:23:05.729835 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:05.729771 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:06.661118 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:06.661075 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" event={"ID":"990d6988-42df-4af3-bec7-4b652a019f99","Type":"ContainerStarted","Data":"86b3ef34862e0af1fb2e40fbd8da31e6e3a75b1cae4b6651e4d5237ee63980d2"} Jul 08 18:23:06.661578 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:06.661159 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" event={"ID":"990d6988-42df-4af3-bec7-4b652a019f99","Type":"ContainerStarted","Data":"b163643268fb8e8402159ef4006545fe1e87e93035bb684c826a10761e90641c"} Jul 08 18:23:06.661578 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:06.661295 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:06.685618 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:06.685572 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" podStartSLOduration=2.8541918219999998 podStartE2EDuration="6.685554908s" totalImagesPullingTime="3.831363086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:23:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:23:06.683324325 +0000 UTC m=+307.434633798" watchObservedRunningTime="2026-07-08 18:23:06.685554908 +0000 UTC m=+307.436864435" Jul 08 18:23:06.764425 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:06.764400 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:23:07.667152 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:07.667111 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" event={"ID":"1f231c29-f57a-48b2-b71e-c5c2f80be098","Type":"ContainerStarted","Data":"8d1f7c6c0d37131774f90a8748494fd60eb3c2ba8201e1727f59f79f7da0cace"} Jul 08 18:23:07.667627 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:07.667179 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" event={"ID":"1f231c29-f57a-48b2-b71e-c5c2f80be098","Type":"ContainerStarted","Data":"05feac8b5d7718794bfa6c1a7b87c5275bfca377b628e596256ba92078c7b284"} Jul 08 18:23:07.667627 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:07.667204 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" event={"ID":"1f231c29-f57a-48b2-b71e-c5c2f80be098","Type":"ContainerStarted","Data":"a7c04668bf71b880556b46532e879af6a35f83a7aa342b20682e92e71911d4ba"} Jul 08 18:23:07.668723 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:07.668701 2574 generic.go:356] "Generic (PLEG): container finished" podID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerID="a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab" exitCode=0 Jul 08 18:23:07.668824 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:07.668739 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"08b800d1-45ed-460a-b934-6f76d00d2f6d","Type":"ContainerDied","Data":"a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab"} Jul 08 18:23:07.684218 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:07.684186 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:07.700124 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:07.700045 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-859bf86486-s6rv4" podStartSLOduration=2.204390232 podStartE2EDuration="4.700025507s" totalImagesPullingTime="2.495635275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:23:07.69923096 +0000 UTC m=+308.450540434" watchObservedRunningTime="2026-07-08 18:23:07.700025507 +0000 UTC m=+308.451334983" Jul 08 18:23:12.675780 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:12.675746 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7d868f677b-hlglt" Jul 08 18:23:13.696435 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:13.696401 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"08b800d1-45ed-460a-b934-6f76d00d2f6d","Type":"ContainerStarted","Data":"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a"} Jul 08 18:23:13.696827 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:13.696466 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"08b800d1-45ed-460a-b934-6f76d00d2f6d","Type":"ContainerStarted","Data":"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f"} Jul 08 18:23:13.696827 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:13.696494 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"08b800d1-45ed-460a-b934-6f76d00d2f6d","Type":"ContainerStarted","Data":"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294"} Jul 08 18:23:13.696827 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:13.696519 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"08b800d1-45ed-460a-b934-6f76d00d2f6d","Type":"ContainerStarted","Data":"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55"} Jul 08 18:23:13.696827 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:13.696546 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"08b800d1-45ed-460a-b934-6f76d00d2f6d","Type":"ContainerStarted","Data":"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85"} Jul 08 18:23:13.696827 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:13.696569 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"08b800d1-45ed-460a-b934-6f76d00d2f6d","Type":"ContainerStarted","Data":"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044"} Jul 08 18:23:13.746474 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:13.746413 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.99828544 podStartE2EDuration="9.746392552s" totalImagesPullingTime="6.730227991s" totalInitContainerRuntime="17.879121ms" isStatefulPod=false podCreationTimestamp="2026-07-08 18:23:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:23:13.744749389 +0000 UTC m=+314.496058865" watchObservedRunningTime="2026-07-08 18:23:13.746392552 +0000 UTC m=+314.497702028" Jul 08 18:23:15.122421 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:15.122392 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:23:16.697606 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:16.697574 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:16.756835 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:16.756805 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:17.116453 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:17.116425 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:17.175612 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:17.175580 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:20.376593 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:20.376561 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:20.452919 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:20.452875 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:26.746450 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:26.746365 2574 generic.go:356] "Generic (PLEG): container finished" podID="07b57a1f-a5d2-4bfa-81ac-188e4c33816d" containerID="e2539cfc657dc63353606ad5785f1c866ab575c54875b84b5f364c82aa951d10" exitCode=0 Jul 08 18:23:26.746800 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:26.746440 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" event={"ID":"07b57a1f-a5d2-4bfa-81ac-188e4c33816d","Type":"ContainerDied","Data":"e2539cfc657dc63353606ad5785f1c866ab575c54875b84b5f364c82aa951d10"} Jul 08 18:23:26.746861 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:26.746799 2574 scope.go:117] "RemoveContainer" containerID="e2539cfc657dc63353606ad5785f1c866ab575c54875b84b5f364c82aa951d10" Jul 08 18:23:27.751609 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:27.751577 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-qkmvz" event={"ID":"07b57a1f-a5d2-4bfa-81ac-188e4c33816d","Type":"ContainerStarted","Data":"14c4f5ca06cd7fe689234f0e70e008f487052844ecd6cf2ae2c4acaf5150e035"} Jul 08 18:23:27.752992 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:27.752970 2574 generic.go:356] "Generic (PLEG): container finished" podID="fefa4c54-c8d3-48ee-97fd-6358949e39cf" containerID="511b5988d2d973125880169f8f0ae90cab888918aef9d288b0ded4e92858a1d8" exitCode=0 Jul 08 18:23:27.753103 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:27.753022 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np" event={"ID":"fefa4c54-c8d3-48ee-97fd-6358949e39cf","Type":"ContainerDied","Data":"511b5988d2d973125880169f8f0ae90cab888918aef9d288b0ded4e92858a1d8"} Jul 08 18:23:27.753291 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:27.753279 2574 scope.go:117] "RemoveContainer" containerID="511b5988d2d973125880169f8f0ae90cab888918aef9d288b0ded4e92858a1d8" Jul 08 18:23:28.758655 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:28.758625 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t58np" event={"ID":"fefa4c54-c8d3-48ee-97fd-6358949e39cf","Type":"ContainerStarted","Data":"2a4a9e57039d059f5f0cbd8387e2c0ce79fed2a86f5faa9044ba83352958e81b"} Jul 08 18:23:35.386314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:35.386262 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:35.483561 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:35.483527 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:42.119229 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:42.119196 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:42.219808 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:42.219773 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:23:42.274991 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:23:42.274956 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:01.361297 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:01.361259 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:01.461644 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:01.461619 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:05.122472 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:05.122437 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:05.138686 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:05.138661 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:05.909884 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:05.909840 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:11.781070 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:11.781035 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:11.879287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:11.879252 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:11.932288 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:11.932256 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:12.059488 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:12.059406 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:12.238825 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:12.238794 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:12.450206 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:12.450175 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:12.501469 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:12.501445 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:13.448785 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:13.448728 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:13.552693 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:13.552648 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs\") pod \"network-metrics-daemon-ts8sd\" (UID: \"0d868453-b10b-4dd9-b843-91de3f1b48d3\") " pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:24:13.555014 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:13.554992 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d868453-b10b-4dd9-b843-91de3f1b48d3-metrics-certs\") pod \"network-metrics-daemon-ts8sd\" (UID: \"0d868453-b10b-4dd9-b843-91de3f1b48d3\") " pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:24:13.668055 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:13.668024 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9qsc4\"" Jul 08 18:24:13.670534 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:13.670513 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:13.676424 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:13.676410 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ts8sd" Jul 08 18:24:13.738247 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:13.738217 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:13.806415 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:13.806386 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ts8sd"] Jul 08 18:24:13.809060 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:24:13.809018 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d868453_b10b_4dd9_b843_91de3f1b48d3.slice/crio-e31a630294c47a528b3c0d4f3ff080eba6e8c1c52c9f4ed355daba6d49c1d1ad WatchSource:0}: Error finding container e31a630294c47a528b3c0d4f3ff080eba6e8c1c52c9f4ed355daba6d49c1d1ad: Status 404 returned error can't find the container with id e31a630294c47a528b3c0d4f3ff080eba6e8c1c52c9f4ed355daba6d49c1d1ad Jul 08 18:24:13.843298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:13.843269 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:13.925967 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:13.925932 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ts8sd" event={"ID":"0d868453-b10b-4dd9-b843-91de3f1b48d3","Type":"ContainerStarted","Data":"e31a630294c47a528b3c0d4f3ff080eba6e8c1c52c9f4ed355daba6d49c1d1ad"} Jul 08 18:24:14.048497 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:14.048395 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:14.146872 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:14.146822 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:15.936830 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:15.936801 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ts8sd" event={"ID":"0d868453-b10b-4dd9-b843-91de3f1b48d3","Type":"ContainerStarted","Data":"a481eb27cb3a5119c2c929bee8040decbf1b46e891b1d5c7360ce98b24bf66ac"} Jul 08 18:24:15.936830 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:15.936861 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ts8sd" event={"ID":"0d868453-b10b-4dd9-b843-91de3f1b48d3","Type":"ContainerStarted","Data":"f634c836062f06309a3b1360aeafc784e3a82a6b5e52d94aa895ec6069ae8d14"} Jul 08 18:24:15.954887 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:15.954829 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ts8sd" podStartSLOduration=375.855706864 podStartE2EDuration="6m16.954814404s" totalImagesPullingTime="1.09910754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:24:15.952953491 +0000 UTC m=+376.704262965" watchObservedRunningTime="2026-07-08 18:24:15.954814404 +0000 UTC m=+376.706123877" Jul 08 18:24:16.046218 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:16.046185 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:16.103742 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:16.103707 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:16.162868 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:16.162830 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:21.713513 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:21.713481 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:21.772704 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:21.772676 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:23.698148 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:23.698113 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:23.798280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:23.798252 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:24.462138 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:24.462090 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:24.512991 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:24.512961 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:27.902750 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:27.902710 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:27.956514 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:27.956485 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:34.070918 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:34.070879 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:34.122729 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:34.122695 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:34.674391 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:34.674355 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:34.724920 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:34.724888 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:35.375788 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:35.375750 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:35.474880 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:35.474834 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:35.574430 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:35.574399 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:35.642670 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:35.642582 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:35.939417 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:35.939337 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:38.423207 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:38.423173 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:39.618117 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:39.618073 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:39.719519 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:39.719482 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:40.920406 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:40.920373 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:24:40.920804 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:40.920723 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="prometheus" containerID="cri-o://34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044" gracePeriod=600 Jul 08 18:24:40.920804 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:40.920762 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a" gracePeriod=600 Jul 08 18:24:40.920956 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:40.920782 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="kube-rbac-proxy-web" containerID="cri-o://ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294" gracePeriod=600 Jul 08 18:24:40.920956 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:40.920815 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="kube-rbac-proxy" containerID="cri-o://d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f" gracePeriod=600 Jul 08 18:24:40.920956 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:40.920835 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="thanos-sidecar" containerID="cri-o://d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55" gracePeriod=600 Jul 08 18:24:40.920956 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:40.920873 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="config-reloader" containerID="cri-o://40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85" gracePeriod=600 Jul 08 18:24:41.039445 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.039415 2574 generic.go:356] "Generic (PLEG): container finished" podID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerID="ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294" exitCode=0 Jul 08 18:24:41.039575 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.039491 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"08b800d1-45ed-460a-b934-6f76d00d2f6d","Type":"ContainerDied","Data":"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294"} Jul 08 18:24:41.164739 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.164716 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:41.185790 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.185726 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-thanos-prometheus-http-client-file\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.185790 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.185754 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-grpc-tls\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.185790 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.185777 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.186062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.185795 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-web-config\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.186062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.185826 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-2\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.186062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.185868 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.186062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.185896 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08b800d1-45ed-460a-b934-6f76d00d2f6d-tls-assets\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.186062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.185931 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6f4pv\" (UniqueName: \"kubernetes.io/projected/08b800d1-45ed-460a-b934-6f76d00d2f6d-kube-api-access-6f4pv\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.186062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.185956 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-0\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.186062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.186000 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-tls\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.186062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.186024 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-kube-rbac-proxy\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.186062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.186055 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-serving-certs-ca-bundle\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.186493 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.186119 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-1\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.186493 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.186152 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-kubelet-serving-ca-bundle\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.186493 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.186185 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-db\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.186493 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.186218 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-metrics-client-ca\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.186493 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.186220 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:24:41.186493 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.186250 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-metrics-client-certs\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.186493 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.186275 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08b800d1-45ed-460a-b934-6f76d00d2f6d-config-out\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.186493 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.186299 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-trusted-ca-bundle\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.186493 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.186327 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-config\") pod \"08b800d1-45ed-460a-b934-6f76d00d2f6d\" (UID: \"08b800d1-45ed-460a-b934-6f76d00d2f6d\") " Jul 08 18:24:41.187124 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.186621 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.188309 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.187187 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:24:41.188309 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.187195 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:24:41.188309 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.187503 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:24:41.188309 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.188108 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:24:41.188548 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.188436 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:24:41.188892 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.188864 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:24:41.188996 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.188951 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:24:41.189197 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.189157 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:24:41.189627 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.189588 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:24:41.189838 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.189812 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08b800d1-45ed-460a-b934-6f76d00d2f6d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:24:41.190815 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.190774 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:24:41.191060 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.191037 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:24:41.191433 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.191408 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:24:41.191749 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.191719 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-config" (OuterVolumeSpecName: "config") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:24:41.191940 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.191920 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08b800d1-45ed-460a-b934-6f76d00d2f6d-config-out" (OuterVolumeSpecName: "config-out") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:24:41.192151 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.192125 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08b800d1-45ed-460a-b934-6f76d00d2f6d-kube-api-access-6f4pv" (OuterVolumeSpecName: "kube-api-access-6f4pv") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "kube-api-access-6f4pv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:24:41.192233 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.192210 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:24:41.192518 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.192503 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:24:41.203057 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.203027 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-web-config" (OuterVolumeSpecName: "web-config") pod "08b800d1-45ed-460a-b934-6f76d00d2f6d" (UID: "08b800d1-45ed-460a-b934-6f76d00d2f6d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:24:41.287108 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287072 2574 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-metrics-client-certs\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287108 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287102 2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08b800d1-45ed-460a-b934-6f76d00d2f6d-config-out\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287108 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287114 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287338 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287123 2574 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-config\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287338 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287133 2574 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-thanos-prometheus-http-client-file\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287338 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287142 2574 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-grpc-tls\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287338 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287151 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287338 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287168 2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-web-config\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287338 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287177 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287338 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287185 2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08b800d1-45ed-460a-b934-6f76d00d2f6d-tls-assets\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287338 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287192 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6f4pv\" (UniqueName: \"kubernetes.io/projected/08b800d1-45ed-460a-b934-6f76d00d2f6d-kube-api-access-6f4pv\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287338 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287200 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287338 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287208 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-prometheus-k8s-tls\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287338 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287216 2574 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08b800d1-45ed-460a-b934-6f76d00d2f6d-secret-kube-rbac-proxy\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287338 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287224 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287338 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287232 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287338 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287240 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287338 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287248 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/08b800d1-45ed-460a-b934-6f76d00d2f6d-prometheus-k8s-db\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:41.287338 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:41.287255 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08b800d1-45ed-460a-b934-6f76d00d2f6d-configmap-metrics-client-ca\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:24:42.047142 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.047047 2574 generic.go:356] "Generic (PLEG): container finished" podID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerID="d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a" exitCode=0 Jul 08 18:24:42.047142 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.047073 2574 generic.go:356] "Generic (PLEG): container finished" podID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerID="d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f" exitCode=0 Jul 08 18:24:42.047142 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.047082 2574 generic.go:356] "Generic (PLEG): container finished" podID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerID="d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55" exitCode=0 Jul 08 18:24:42.047142 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.047092 2574 generic.go:356] "Generic (PLEG): container finished" podID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerID="40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85" exitCode=0 Jul 08 18:24:42.047142 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.047101 2574 generic.go:356] "Generic (PLEG): container finished" podID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerID="34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044" exitCode=0 Jul 08 18:24:42.047142 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.047116 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.047142 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.047129 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"08b800d1-45ed-460a-b934-6f76d00d2f6d","Type":"ContainerDied","Data":"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a"} Jul 08 18:24:42.047811 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.047208 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"08b800d1-45ed-460a-b934-6f76d00d2f6d","Type":"ContainerDied","Data":"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f"} Jul 08 18:24:42.047811 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.047237 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"08b800d1-45ed-460a-b934-6f76d00d2f6d","Type":"ContainerDied","Data":"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55"} Jul 08 18:24:42.047811 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.047262 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"08b800d1-45ed-460a-b934-6f76d00d2f6d","Type":"ContainerDied","Data":"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85"} Jul 08 18:24:42.047811 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.047287 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"08b800d1-45ed-460a-b934-6f76d00d2f6d","Type":"ContainerDied","Data":"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044"} Jul 08 18:24:42.047811 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.047299 2574 scope.go:117] "RemoveContainer" containerID="d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a" Jul 08 18:24:42.047811 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.047315 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"08b800d1-45ed-460a-b934-6f76d00d2f6d","Type":"ContainerDied","Data":"d2deb4d516526f136c8607821244d1e83d77a83daa5ff42fa4cc816cdb022ca3"} Jul 08 18:24:42.056424 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.056398 2574 scope.go:117] "RemoveContainer" containerID="d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f" Jul 08 18:24:42.063621 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.063598 2574 scope.go:117] "RemoveContainer" containerID="ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294" Jul 08 18:24:42.068130 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.068107 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:24:42.071520 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.071501 2574 scope.go:117] "RemoveContainer" containerID="d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55" Jul 08 18:24:42.073102 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.073082 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:24:42.078491 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.078474 2574 scope.go:117] "RemoveContainer" containerID="40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85" Jul 08 18:24:42.085034 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.085018 2574 scope.go:117] "RemoveContainer" containerID="34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044" Jul 08 18:24:42.092425 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.092408 2574 scope.go:117] "RemoveContainer" containerID="a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab" Jul 08 18:24:42.098430 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098407 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:24:42.098712 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098700 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="kube-rbac-proxy-web" Jul 08 18:24:42.098750 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098714 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="kube-rbac-proxy-web" Jul 08 18:24:42.098750 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098722 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="thanos-sidecar" Jul 08 18:24:42.098750 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098728 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="thanos-sidecar" Jul 08 18:24:42.098750 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098737 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="kube-rbac-proxy-thanos" Jul 08 18:24:42.098750 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098742 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="kube-rbac-proxy-thanos" Jul 08 18:24:42.098921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098760 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="kube-rbac-proxy" Jul 08 18:24:42.098921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098767 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="kube-rbac-proxy" Jul 08 18:24:42.098921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098774 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="init-config-reloader" Jul 08 18:24:42.098921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098778 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="init-config-reloader" Jul 08 18:24:42.098921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098789 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="config-reloader" Jul 08 18:24:42.098921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098794 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="config-reloader" Jul 08 18:24:42.098921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098800 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="prometheus" Jul 08 18:24:42.098921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098805 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="prometheus" Jul 08 18:24:42.098921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098869 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="kube-rbac-proxy-thanos" Jul 08 18:24:42.098921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098878 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="kube-rbac-proxy-web" Jul 08 18:24:42.098921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098886 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="prometheus" Jul 08 18:24:42.098921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098891 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="thanos-sidecar" Jul 08 18:24:42.098921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098897 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="config-reloader" Jul 08 18:24:42.098921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.098903 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" containerName="kube-rbac-proxy" Jul 08 18:24:42.101305 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.101290 2574 scope.go:117] "RemoveContainer" containerID="d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a" Jul 08 18:24:42.101616 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:24:42.101586 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a\": container with ID starting with d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a not found: ID does not exist" containerID="d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a" Jul 08 18:24:42.101706 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.101622 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a"} err="failed to get container status \"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a\": rpc error: code = NotFound desc = could not find container \"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a\": container with ID starting with d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a not found: ID does not exist" Jul 08 18:24:42.101706 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.101655 2574 scope.go:117] "RemoveContainer" containerID="d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f" Jul 08 18:24:42.101915 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:24:42.101899 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f\": container with ID starting with d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f not found: ID does not exist" containerID="d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f" Jul 08 18:24:42.101961 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.101922 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f"} err="failed to get container status \"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f\": rpc error: code = NotFound desc = could not find container \"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f\": container with ID starting with d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f not found: ID does not exist" Jul 08 18:24:42.101961 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.101935 2574 scope.go:117] "RemoveContainer" containerID="ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294" Jul 08 18:24:42.102182 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:24:42.102167 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294\": container with ID starting with ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294 not found: ID does not exist" containerID="ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294" Jul 08 18:24:42.102226 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.102184 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294"} err="failed to get container status \"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294\": rpc error: code = NotFound desc = could not find container \"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294\": container with ID starting with ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294 not found: ID does not exist" Jul 08 18:24:42.102226 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.102196 2574 scope.go:117] "RemoveContainer" containerID="d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55" Jul 08 18:24:42.102415 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:24:42.102396 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55\": container with ID starting with d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55 not found: ID does not exist" containerID="d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55" Jul 08 18:24:42.102454 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.102421 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55"} err="failed to get container status \"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55\": rpc error: code = NotFound desc = could not find container \"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55\": container with ID starting with d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55 not found: ID does not exist" Jul 08 18:24:42.102454 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.102436 2574 scope.go:117] "RemoveContainer" containerID="40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85" Jul 08 18:24:42.102657 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:24:42.102641 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85\": container with ID starting with 40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85 not found: ID does not exist" containerID="40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85" Jul 08 18:24:42.102716 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.102660 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85"} err="failed to get container status \"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85\": rpc error: code = NotFound desc = could not find container \"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85\": container with ID starting with 40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85 not found: ID does not exist" Jul 08 18:24:42.102716 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.102675 2574 scope.go:117] "RemoveContainer" containerID="34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044" Jul 08 18:24:42.102957 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:24:42.102929 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044\": container with ID starting with 34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044 not found: ID does not exist" containerID="34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044" Jul 08 18:24:42.103049 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.102960 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044"} err="failed to get container status \"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044\": rpc error: code = NotFound desc = could not find container \"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044\": container with ID starting with 34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044 not found: ID does not exist" Jul 08 18:24:42.103049 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.102977 2574 scope.go:117] "RemoveContainer" containerID="a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab" Jul 08 18:24:42.103206 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:24:42.103188 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab\": container with ID starting with a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab not found: ID does not exist" containerID="a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab" Jul 08 18:24:42.103249 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.103207 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab"} err="failed to get container status \"a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab\": rpc error: code = NotFound desc = could not find container \"a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab\": container with ID starting with a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab not found: ID does not exist" Jul 08 18:24:42.103249 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.103219 2574 scope.go:117] "RemoveContainer" containerID="d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a" Jul 08 18:24:42.103408 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.103394 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a"} err="failed to get container status \"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a\": rpc error: code = NotFound desc = could not find container \"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a\": container with ID starting with d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a not found: ID does not exist" Jul 08 18:24:42.103444 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.103410 2574 scope.go:117] "RemoveContainer" containerID="d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f" Jul 08 18:24:42.103621 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.103604 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f"} err="failed to get container status \"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f\": rpc error: code = NotFound desc = could not find container \"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f\": container with ID starting with d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f not found: ID does not exist" Jul 08 18:24:42.103654 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.103624 2574 scope.go:117] "RemoveContainer" containerID="ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294" Jul 08 18:24:42.103828 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.103812 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294"} err="failed to get container status \"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294\": rpc error: code = NotFound desc = could not find container \"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294\": container with ID starting with ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294 not found: ID does not exist" Jul 08 18:24:42.103979 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.103828 2574 scope.go:117] "RemoveContainer" containerID="d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55" Jul 08 18:24:42.104058 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.104040 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55"} err="failed to get container status \"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55\": rpc error: code = NotFound desc = could not find container \"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55\": container with ID starting with d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55 not found: ID does not exist" Jul 08 18:24:42.104093 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.104059 2574 scope.go:117] "RemoveContainer" containerID="40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85" Jul 08 18:24:42.104256 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.104235 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85"} err="failed to get container status \"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85\": rpc error: code = NotFound desc = could not find container \"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85\": container with ID starting with 40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85 not found: ID does not exist" Jul 08 18:24:42.104323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.104259 2574 scope.go:117] "RemoveContainer" containerID="34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044" Jul 08 18:24:42.104469 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.104452 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044"} err="failed to get container status \"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044\": rpc error: code = NotFound desc = could not find container \"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044\": container with ID starting with 34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044 not found: ID does not exist" Jul 08 18:24:42.104469 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.104468 2574 scope.go:117] "RemoveContainer" containerID="a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab" Jul 08 18:24:42.104605 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.104584 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.104694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.104668 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab"} err="failed to get container status \"a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab\": rpc error: code = NotFound desc = could not find container \"a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab\": container with ID starting with a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab not found: ID does not exist" Jul 08 18:24:42.104694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.104683 2574 scope.go:117] "RemoveContainer" containerID="d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a" Jul 08 18:24:42.105002 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.104972 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a"} err="failed to get container status \"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a\": rpc error: code = NotFound desc = could not find container \"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a\": container with ID starting with d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a not found: ID does not exist" Jul 08 18:24:42.105002 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.105002 2574 scope.go:117] "RemoveContainer" containerID="d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f" Jul 08 18:24:42.105265 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.105246 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f"} err="failed to get container status \"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f\": rpc error: code = NotFound desc = could not find container \"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f\": container with ID starting with d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f not found: ID does not exist" Jul 08 18:24:42.105265 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.105264 2574 scope.go:117] "RemoveContainer" containerID="ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294" Jul 08 18:24:42.105490 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.105470 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294"} err="failed to get container status \"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294\": rpc error: code = NotFound desc = could not find container \"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294\": container with ID starting with ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294 not found: ID does not exist" Jul 08 18:24:42.105490 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.105490 2574 scope.go:117] "RemoveContainer" containerID="d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55" Jul 08 18:24:42.105713 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.105691 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55"} err="failed to get container status \"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55\": rpc error: code = NotFound desc = could not find container \"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55\": container with ID starting with d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55 not found: ID does not exist" Jul 08 18:24:42.105768 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.105716 2574 scope.go:117] "RemoveContainer" containerID="40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85" Jul 08 18:24:42.105960 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.105941 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85"} err="failed to get container status \"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85\": rpc error: code = NotFound desc = could not find container \"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85\": container with ID starting with 40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85 not found: ID does not exist" Jul 08 18:24:42.105960 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.105959 2574 scope.go:117] "RemoveContainer" containerID="34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044" Jul 08 18:24:42.106199 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.106181 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044"} err="failed to get container status \"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044\": rpc error: code = NotFound desc = could not find container \"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044\": container with ID starting with 34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044 not found: ID does not exist" Jul 08 18:24:42.106249 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.106201 2574 scope.go:117] "RemoveContainer" containerID="a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab" Jul 08 18:24:42.106430 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.106413 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab"} err="failed to get container status \"a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab\": rpc error: code = NotFound desc = could not find container \"a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab\": container with ID starting with a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab not found: ID does not exist" Jul 08 18:24:42.106469 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.106431 2574 scope.go:117] "RemoveContainer" containerID="d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a" Jul 08 18:24:42.106650 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.106615 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a"} err="failed to get container status \"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a\": rpc error: code = NotFound desc = could not find container \"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a\": container with ID starting with d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a not found: ID does not exist" Jul 08 18:24:42.106711 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.106652 2574 scope.go:117] "RemoveContainer" containerID="d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f" Jul 08 18:24:42.106883 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.106864 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f"} err="failed to get container status \"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f\": rpc error: code = NotFound desc = could not find container \"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f\": container with ID starting with d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f not found: ID does not exist" Jul 08 18:24:42.106947 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.106883 2574 scope.go:117] "RemoveContainer" containerID="ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294" Jul 08 18:24:42.107107 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.107085 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294"} err="failed to get container status \"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294\": rpc error: code = NotFound desc = could not find container \"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294\": container with ID starting with ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294 not found: ID does not exist" Jul 08 18:24:42.107185 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.107110 2574 scope.go:117] "RemoveContainer" containerID="d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55" Jul 08 18:24:42.107376 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.107352 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55"} err="failed to get container status \"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55\": rpc error: code = NotFound desc = could not find container \"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55\": container with ID starting with d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55 not found: ID does not exist" Jul 08 18:24:42.107424 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.107379 2574 scope.go:117] "RemoveContainer" containerID="40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85" Jul 08 18:24:42.107515 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.107501 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9bt64jlnnr16n\"" Jul 08 18:24:42.107701 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.107674 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85"} err="failed to get container status \"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85\": rpc error: code = NotFound desc = could not find container \"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85\": container with ID starting with 40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85 not found: ID does not exist" Jul 08 18:24:42.107768 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.107703 2574 scope.go:117] "RemoveContainer" containerID="34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044" Jul 08 18:24:42.107832 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.107765 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 18:24:42.108076 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108045 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044"} err="failed to get container status \"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044\": rpc error: code = NotFound desc = could not find container \"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044\": container with ID starting with 34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044 not found: ID does not exist" Jul 08 18:24:42.108147 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108083 2574 scope.go:117] "RemoveContainer" containerID="a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab" Jul 08 18:24:42.108147 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108063 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 18:24:42.108237 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108164 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 18:24:42.108237 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108210 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 18:24:42.108237 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108232 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 18:24:42.108382 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108287 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 18:24:42.108382 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108212 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 18:24:42.108382 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108348 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 18:24:42.108587 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108572 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 18:24:42.108699 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108676 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab"} err="failed to get container status \"a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab\": rpc error: code = NotFound desc = could not find container \"a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab\": container with ID starting with a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab not found: ID does not exist" Jul 08 18:24:42.108699 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108699 2574 scope.go:117] "RemoveContainer" containerID="d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a" Jul 08 18:24:42.108874 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108752 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xnqfj\"" Jul 08 18:24:42.108874 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108800 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 18:24:42.108874 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108836 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 18:24:42.109037 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108841 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 18:24:42.109037 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108891 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 18:24:42.109037 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.108948 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a"} err="failed to get container status \"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a\": rpc error: code = NotFound desc = could not find container \"d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a\": container with ID starting with d80c84d043d740de6b4a4e4173f3a4c316e1b80d53bacfd5479a968b444ad26a not found: ID does not exist" Jul 08 18:24:42.109037 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.109021 2574 scope.go:117] "RemoveContainer" containerID="d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f" Jul 08 18:24:42.109290 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.109269 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f"} err="failed to get container status \"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f\": rpc error: code = NotFound desc = could not find container \"d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f\": container with ID starting with d0043b7b196d5637fb8377eea713f4a7e62320300f2ef24d98662c34cb87ce9f not found: ID does not exist" Jul 08 18:24:42.109345 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.109292 2574 scope.go:117] "RemoveContainer" containerID="ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294" Jul 08 18:24:42.109606 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.109571 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294"} err="failed to get container status \"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294\": rpc error: code = NotFound desc = could not find container \"ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294\": container with ID starting with ed06693a51f8c03851119a49c62298673742c6d9b719a6983d35b9713829d294 not found: ID does not exist" Jul 08 18:24:42.109606 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.109598 2574 scope.go:117] "RemoveContainer" containerID="d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55" Jul 08 18:24:42.109944 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.109922 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55"} err="failed to get container status \"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55\": rpc error: code = NotFound desc = could not find container \"d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55\": container with ID starting with d50977f8cfbdc084f315a048dac48ac64a79973ca0f2af47bd3b3445c7824f55 not found: ID does not exist" Jul 08 18:24:42.110040 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.109946 2574 scope.go:117] "RemoveContainer" containerID="40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85" Jul 08 18:24:42.110239 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.110219 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85"} err="failed to get container status \"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85\": rpc error: code = NotFound desc = could not find container \"40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85\": container with ID starting with 40fe3a7e1ac24f822c9310276c8d4ba8949bd0b78093890a62d587662c8aaa85 not found: ID does not exist" Jul 08 18:24:42.110239 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.110238 2574 scope.go:117] "RemoveContainer" containerID="34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044" Jul 08 18:24:42.110525 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.110487 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044"} err="failed to get container status \"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044\": rpc error: code = NotFound desc = could not find container \"34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044\": container with ID starting with 34dfc39fa5147679c0f63c15162ca18a15cae802eb973542bcfd69aed933a044 not found: ID does not exist" Jul 08 18:24:42.110525 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.110509 2574 scope.go:117] "RemoveContainer" containerID="a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab" Jul 08 18:24:42.110793 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.110770 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab"} err="failed to get container status \"a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab\": rpc error: code = NotFound desc = could not find container \"a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab\": container with ID starting with a4cf429b80980c4823c12146c13ffdb8b34899ebb563c80aec5d5cc5287e5dab not found: ID does not exist" Jul 08 18:24:42.111956 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.111918 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 18:24:42.113109 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.113090 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 18:24:42.117743 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.117723 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:24:42.193497 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193460 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.193497 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193501 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.193715 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193529 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.193715 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193584 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.193715 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193637 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.193715 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193662 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.193715 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193681 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.193715 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193704 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.193925 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193743 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.193925 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193769 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.193925 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193787 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-config\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.193925 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193809 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.193925 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193826 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.193925 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193872 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.193925 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193887 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.193925 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193904 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sww7\" (UniqueName: \"kubernetes.io/projected/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-kube-api-access-6sww7\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.193925 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193918 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.194181 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193936 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.194181 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193950 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.194181 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.193970 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.294872 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.294820 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295002 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.294886 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295067 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295041 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295100 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295088 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295137 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295123 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295215 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295192 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295350 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295229 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295350 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295259 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-config\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295350 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295284 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295350 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295312 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295350 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295343 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295608 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295367 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295608 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295399 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sww7\" (UniqueName: \"kubernetes.io/projected/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-kube-api-access-6sww7\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295608 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295425 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295608 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295455 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295608 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295463 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295608 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295479 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295608 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295523 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295608 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295569 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.295608 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295597 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.296231 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295626 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.297191 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.296527 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.297191 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.296558 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.297191 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.296801 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.298168 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.298138 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.299414 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.298462 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.299414 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.298556 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.299414 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.298720 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.299414 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.299123 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.299414 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.299325 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.299414 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.295455 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.299414 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.299352 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.299868 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.299824 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.300095 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.300072 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.300457 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.300435 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.300813 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.300790 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.301230 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.301210 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.301546 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.301530 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-config\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.303130 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.303109 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.309786 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.309768 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sww7\" (UniqueName: \"kubernetes.io/projected/2f2c60e0-cbc3-48b8-bc89-74c63d74717a-kube-api-access-6sww7\") pod \"prometheus-k8s-0\" (UID: \"2f2c60e0-cbc3-48b8-bc89-74c63d74717a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.416288 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.416236 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:42.586550 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:42.586521 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:24:42.589506 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:24:42.589482 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f2c60e0_cbc3_48b8_bc89_74c63d74717a.slice/crio-4a4f9f59f2b23b60fc8455fc21c3c978072e62fd3f158117d749f13cf604049c WatchSource:0}: Error finding container 4a4f9f59f2b23b60fc8455fc21c3c978072e62fd3f158117d749f13cf604049c: Status 404 returned error can't find the container with id 4a4f9f59f2b23b60fc8455fc21c3c978072e62fd3f158117d749f13cf604049c Jul 08 18:24:43.052615 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:43.052538 2574 generic.go:356] "Generic (PLEG): container finished" podID="2f2c60e0-cbc3-48b8-bc89-74c63d74717a" containerID="8dcbdb483d269499628ea50a7b5033dd7969f039f179c08e2623382fa892e72e" exitCode=0 Jul 08 18:24:43.053117 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:43.052631 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f2c60e0-cbc3-48b8-bc89-74c63d74717a","Type":"ContainerDied","Data":"8dcbdb483d269499628ea50a7b5033dd7969f039f179c08e2623382fa892e72e"} Jul 08 18:24:43.053117 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:43.052725 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f2c60e0-cbc3-48b8-bc89-74c63d74717a","Type":"ContainerStarted","Data":"4a4f9f59f2b23b60fc8455fc21c3c978072e62fd3f158117d749f13cf604049c"} Jul 08 18:24:43.770456 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:43.770381 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08b800d1-45ed-460a-b934-6f76d00d2f6d" path="/var/lib/kubelet/pods/08b800d1-45ed-460a-b934-6f76d00d2f6d/volumes" Jul 08 18:24:44.064032 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:44.063998 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f2c60e0-cbc3-48b8-bc89-74c63d74717a","Type":"ContainerStarted","Data":"df414719866fca93626fd80f8b9fed565454e34dbce6a9cfd653663c0a9bcdd5"} Jul 08 18:24:44.064501 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:44.064060 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f2c60e0-cbc3-48b8-bc89-74c63d74717a","Type":"ContainerStarted","Data":"4a6ad6eb1e0ad7ecac93cd3119f185c1ac5edec25b0a340bfc2ae058c3297943"} Jul 08 18:24:44.064501 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:44.064085 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f2c60e0-cbc3-48b8-bc89-74c63d74717a","Type":"ContainerStarted","Data":"6b4902c5ca0b5eb4faa1537e26887e681a10217ffe7707d9bae4467ded8fec0d"} Jul 08 18:24:44.064501 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:44.064109 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f2c60e0-cbc3-48b8-bc89-74c63d74717a","Type":"ContainerStarted","Data":"30ba0970b9e7afd30a1ea6f9b5944b2bd1bea73ace01b15e4f93ea00e652a3e8"} Jul 08 18:24:44.064501 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:44.064133 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f2c60e0-cbc3-48b8-bc89-74c63d74717a","Type":"ContainerStarted","Data":"73525c8a8d25b22322f23ae23ca50b11bf606204997ebdb5933c98f51e12cc4f"} Jul 08 18:24:44.064501 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:44.064156 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f2c60e0-cbc3-48b8-bc89-74c63d74717a","Type":"ContainerStarted","Data":"dab1e49182b478a4ad47e220bc275cf92c951b8d9fd4b955f284daf1c0f75a2b"} Jul 08 18:24:44.098244 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:44.098190 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.078986599 podStartE2EDuration="2.098169079s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.18248ms" isStatefulPod=false podCreationTimestamp="2026-07-08 18:24:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:24:44.095621495 +0000 UTC m=+404.846930968" watchObservedRunningTime="2026-07-08 18:24:44.098169079 +0000 UTC m=+404.849478552" Jul 08 18:24:47.416438 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:47.416401 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:24:50.397136 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:50.397101 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:50.495429 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:24:50.495395 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:25:17.382016 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:25:17.381963 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:25:17.438309 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:25:17.438279 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:25:26.577787 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:25:26.577705 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:25:26.679038 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:25:26.679002 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:25:42.417510 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:25:42.417471 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:25:42.433249 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:25:42.433222 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:25:43.306937 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:25:43.306909 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:25:53.232597 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:25:53.232564 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:25:54.069021 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:25:54.068980 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:25:56.853519 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:25:56.853471 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:25:56.952127 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:25:56.952101 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:25:58.771285 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:25:58.771251 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:25:58.872840 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:25:58.872814 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:25:59.301407 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:25:59.301373 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:25:59.406200 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:25:59.406173 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:00.782937 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:00.782904 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:01.170839 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:01.170813 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:03.849937 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:03.849906 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:03.950239 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:03.950202 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:06.969109 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:06.969077 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:07.070148 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:07.070120 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:09.387041 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:09.387000 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:15.453171 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:15.453138 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:15.552881 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:15.552834 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:20.672722 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:20.672689 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:20.772626 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:20.772598 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:22.771350 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:22.771313 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:35.266230 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:35.266194 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:35.362361 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:35.362329 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:35.920288 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:35.920253 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:36.156736 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.156704 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:36.253795 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.253713 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:36.313606 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.313566 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7"] Jul 08 18:26:36.317120 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.317098 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:36.319652 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.319623 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 18:26:36.319769 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.319665 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 18:26:36.319933 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.319919 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q6glk\"" Jul 08 18:26:36.320192 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.320171 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 18:26:36.320259 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.320221 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 18:26:36.333570 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.333549 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7"] Jul 08 18:26:36.355995 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.355968 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5494476b-7149-447d-bf7d-c28859c1f24b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-578597fb9c-5pzd7\" (UID: \"5494476b-7149-447d-bf7d-c28859c1f24b\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:36.356094 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.356000 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5494476b-7149-447d-bf7d-c28859c1f24b-webhook-cert\") pod \"opendatahub-operator-controller-manager-578597fb9c-5pzd7\" (UID: \"5494476b-7149-447d-bf7d-c28859c1f24b\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:36.356094 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.356034 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5494476b-7149-447d-bf7d-c28859c1f24b-manifests\") pod \"opendatahub-operator-controller-manager-578597fb9c-5pzd7\" (UID: \"5494476b-7149-447d-bf7d-c28859c1f24b\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:36.356094 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.356050 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5494476b-7149-447d-bf7d-c28859c1f24b-tmp\") pod \"opendatahub-operator-controller-manager-578597fb9c-5pzd7\" (UID: \"5494476b-7149-447d-bf7d-c28859c1f24b\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:36.356238 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.356101 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjvcl\" (UniqueName: \"kubernetes.io/projected/5494476b-7149-447d-bf7d-c28859c1f24b-kube-api-access-zjvcl\") pod \"opendatahub-operator-controller-manager-578597fb9c-5pzd7\" (UID: \"5494476b-7149-447d-bf7d-c28859c1f24b\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:36.430951 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.430921 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:36.457253 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.457228 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5494476b-7149-447d-bf7d-c28859c1f24b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-578597fb9c-5pzd7\" (UID: \"5494476b-7149-447d-bf7d-c28859c1f24b\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:36.457345 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.457263 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5494476b-7149-447d-bf7d-c28859c1f24b-webhook-cert\") pod \"opendatahub-operator-controller-manager-578597fb9c-5pzd7\" (UID: \"5494476b-7149-447d-bf7d-c28859c1f24b\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:36.457345 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.457296 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5494476b-7149-447d-bf7d-c28859c1f24b-manifests\") pod \"opendatahub-operator-controller-manager-578597fb9c-5pzd7\" (UID: \"5494476b-7149-447d-bf7d-c28859c1f24b\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:36.457345 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.457312 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5494476b-7149-447d-bf7d-c28859c1f24b-tmp\") pod \"opendatahub-operator-controller-manager-578597fb9c-5pzd7\" (UID: \"5494476b-7149-447d-bf7d-c28859c1f24b\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:36.457345 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.457335 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjvcl\" (UniqueName: \"kubernetes.io/projected/5494476b-7149-447d-bf7d-c28859c1f24b-kube-api-access-zjvcl\") pod \"opendatahub-operator-controller-manager-578597fb9c-5pzd7\" (UID: \"5494476b-7149-447d-bf7d-c28859c1f24b\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:36.457763 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.457729 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5494476b-7149-447d-bf7d-c28859c1f24b-manifests\") pod \"opendatahub-operator-controller-manager-578597fb9c-5pzd7\" (UID: \"5494476b-7149-447d-bf7d-c28859c1f24b\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:36.459789 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.459770 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5494476b-7149-447d-bf7d-c28859c1f24b-tmp\") pod \"opendatahub-operator-controller-manager-578597fb9c-5pzd7\" (UID: \"5494476b-7149-447d-bf7d-c28859c1f24b\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:36.459935 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.459917 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5494476b-7149-447d-bf7d-c28859c1f24b-webhook-cert\") pod \"opendatahub-operator-controller-manager-578597fb9c-5pzd7\" (UID: \"5494476b-7149-447d-bf7d-c28859c1f24b\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:36.460019 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.460004 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5494476b-7149-447d-bf7d-c28859c1f24b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-578597fb9c-5pzd7\" (UID: \"5494476b-7149-447d-bf7d-c28859c1f24b\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:36.465759 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.465731 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjvcl\" (UniqueName: \"kubernetes.io/projected/5494476b-7149-447d-bf7d-c28859c1f24b-kube-api-access-zjvcl\") pod \"opendatahub-operator-controller-manager-578597fb9c-5pzd7\" (UID: \"5494476b-7149-447d-bf7d-c28859c1f24b\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:36.532611 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.532534 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:36.628054 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.628017 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:36.631976 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.631943 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:36.736630 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.736596 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:36.779153 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.779129 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7"] Jul 08 18:26:36.781679 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:26:36.781654 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5494476b_7149_447d_bf7d_c28859c1f24b.slice/crio-1b21065bc682a8bf0a205fefa574523c9e872ed570f2432eb8b28bf02b1d6a2c WatchSource:0}: Error finding container 1b21065bc682a8bf0a205fefa574523c9e872ed570f2432eb8b28bf02b1d6a2c: Status 404 returned error can't find the container with id 1b21065bc682a8bf0a205fefa574523c9e872ed570f2432eb8b28bf02b1d6a2c Jul 08 18:26:36.835972 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:36.835938 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:37.031554 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:37.031512 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:37.147278 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:37.147247 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:37.500706 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:37.500623 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" event={"ID":"5494476b-7149-447d-bf7d-c28859c1f24b","Type":"ContainerStarted","Data":"1b21065bc682a8bf0a205fefa574523c9e872ed570f2432eb8b28bf02b1d6a2c"} Jul 08 18:26:40.531818 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:40.531780 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" event={"ID":"5494476b-7149-447d-bf7d-c28859c1f24b","Type":"ContainerStarted","Data":"a90fa7cdfde59445d717ad864fa7c93e87d7f3c3e6c3d8368ed44d5ec7e75b5f"} Jul 08 18:26:41.537576 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:41.537538 2574 generic.go:356] "Generic (PLEG): container finished" podID="5494476b-7149-447d-bf7d-c28859c1f24b" containerID="a90fa7cdfde59445d717ad864fa7c93e87d7f3c3e6c3d8368ed44d5ec7e75b5f" exitCode=0 Jul 08 18:26:41.537576 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:41.537574 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" event={"ID":"5494476b-7149-447d-bf7d-c28859c1f24b","Type":"ContainerDied","Data":"a90fa7cdfde59445d717ad864fa7c93e87d7f3c3e6c3d8368ed44d5ec7e75b5f"} Jul 08 18:26:42.543673 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:42.543638 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" event={"ID":"5494476b-7149-447d-bf7d-c28859c1f24b","Type":"ContainerStarted","Data":"e15d10148382e2c4f04d1269da094cf7cd8bfaefc247a1fade9855ce8705b29d"} Jul 08 18:26:42.544159 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:42.543751 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:42.573175 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:42.573125 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" podStartSLOduration=2.032580964 podStartE2EDuration="6.57310801s" totalImagesPullingTime="3.465580637s" totalInitContainerRuntime="1.074946409s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:26:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:26:42.570924487 +0000 UTC m=+523.322233962" watchObservedRunningTime="2026-07-08 18:26:42.57310801 +0000 UTC m=+523.324417484" Jul 08 18:26:47.424144 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:47.424109 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:47.536410 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:47.536369 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:53.549981 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:53.549951 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-5pzd7" Jul 08 18:26:55.109830 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.109793 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:55.216273 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.216241 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:55.541350 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.541263 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:55.674813 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.674780 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:55.786467 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.786423 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv"] Jul 08 18:26:55.797414 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.797350 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" Jul 08 18:26:55.801041 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.801020 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 18:26:55.802997 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.802951 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 18:26:55.803186 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.803170 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 18:26:55.803300 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.803187 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-k578t\"" Jul 08 18:26:55.805143 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.804979 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:26:55.807922 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.807901 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 18:26:55.820633 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.820612 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv"] Jul 08 18:26:55.931508 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.931478 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq5bx\" (UniqueName: \"kubernetes.io/projected/4925964a-8302-4165-af39-acb1fe708708-kube-api-access-jq5bx\") pod \"lws-controller-manager-56665c6f48-jc6wv\" (UID: \"4925964a-8302-4165-af39-acb1fe708708\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" Jul 08 18:26:55.931663 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.931553 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4925964a-8302-4165-af39-acb1fe708708-metrics-cert\") pod \"lws-controller-manager-56665c6f48-jc6wv\" (UID: \"4925964a-8302-4165-af39-acb1fe708708\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" Jul 08 18:26:55.931663 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.931605 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4925964a-8302-4165-af39-acb1fe708708-manager-config\") pod \"lws-controller-manager-56665c6f48-jc6wv\" (UID: \"4925964a-8302-4165-af39-acb1fe708708\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" Jul 08 18:26:55.931755 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.931702 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4925964a-8302-4165-af39-acb1fe708708-cert\") pod \"lws-controller-manager-56665c6f48-jc6wv\" (UID: \"4925964a-8302-4165-af39-acb1fe708708\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" Jul 08 18:26:55.969381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:55.969348 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:56.032906 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:56.032873 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4925964a-8302-4165-af39-acb1fe708708-manager-config\") pod \"lws-controller-manager-56665c6f48-jc6wv\" (UID: \"4925964a-8302-4165-af39-acb1fe708708\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" Jul 08 18:26:56.033085 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:56.032926 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4925964a-8302-4165-af39-acb1fe708708-cert\") pod \"lws-controller-manager-56665c6f48-jc6wv\" (UID: \"4925964a-8302-4165-af39-acb1fe708708\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" Jul 08 18:26:56.033085 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:56.032958 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq5bx\" (UniqueName: \"kubernetes.io/projected/4925964a-8302-4165-af39-acb1fe708708-kube-api-access-jq5bx\") pod \"lws-controller-manager-56665c6f48-jc6wv\" (UID: \"4925964a-8302-4165-af39-acb1fe708708\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" Jul 08 18:26:56.033085 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:56.032989 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4925964a-8302-4165-af39-acb1fe708708-metrics-cert\") pod \"lws-controller-manager-56665c6f48-jc6wv\" (UID: \"4925964a-8302-4165-af39-acb1fe708708\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" Jul 08 18:26:56.033647 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:56.033625 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4925964a-8302-4165-af39-acb1fe708708-manager-config\") pod \"lws-controller-manager-56665c6f48-jc6wv\" (UID: \"4925964a-8302-4165-af39-acb1fe708708\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" Jul 08 18:26:56.035439 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:56.035420 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4925964a-8302-4165-af39-acb1fe708708-cert\") pod \"lws-controller-manager-56665c6f48-jc6wv\" (UID: \"4925964a-8302-4165-af39-acb1fe708708\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" Jul 08 18:26:56.035935 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:56.035917 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4925964a-8302-4165-af39-acb1fe708708-metrics-cert\") pod \"lws-controller-manager-56665c6f48-jc6wv\" (UID: \"4925964a-8302-4165-af39-acb1fe708708\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" Jul 08 18:26:56.040792 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:56.040775 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq5bx\" (UniqueName: \"kubernetes.io/projected/4925964a-8302-4165-af39-acb1fe708708-kube-api-access-jq5bx\") pod \"lws-controller-manager-56665c6f48-jc6wv\" (UID: \"4925964a-8302-4165-af39-acb1fe708708\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" Jul 08 18:26:56.071844 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:56.071814 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:56.107787 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:56.107763 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" Jul 08 18:26:56.238505 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:56.238475 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv"] Jul 08 18:26:56.241921 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:26:56.241891 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4925964a_8302_4165_af39_acb1fe708708.slice/crio-f1901325f8c99d6a54cf94daf6b0a132d02603c58d3b101534e82420bdd21afd WatchSource:0}: Error finding container f1901325f8c99d6a54cf94daf6b0a132d02603c58d3b101534e82420bdd21afd: Status 404 returned error can't find the container with id f1901325f8c99d6a54cf94daf6b0a132d02603c58d3b101534e82420bdd21afd Jul 08 18:26:56.585013 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:56.584982 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:56.602510 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:56.602480 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" event={"ID":"4925964a-8302-4165-af39-acb1fe708708","Type":"ContainerStarted","Data":"f1901325f8c99d6a54cf94daf6b0a132d02603c58d3b101534e82420bdd21afd"} Jul 08 18:26:56.696773 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:56.696742 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:57.067477 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:57.067436 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:57.166061 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:57.166025 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:57.366471 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:57.366359 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:58.112046 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:58.111975 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:26:59.623649 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:59.623617 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" event={"ID":"4925964a-8302-4165-af39-acb1fe708708","Type":"ContainerStarted","Data":"5614843f91ceb68dedf980b654171d8eea2a8bc33f4ef58493a094423a5e9939"} Jul 08 18:26:59.624012 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:59.623696 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" Jul 08 18:26:59.641181 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:26:59.641130 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" podStartSLOduration=1.998606101 podStartE2EDuration="4.641113476s" totalImagesPullingTime="2.642507375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:26:59.640837102 +0000 UTC m=+540.392146579" watchObservedRunningTime="2026-07-08 18:26:59.641113476 +0000 UTC m=+540.392422951" Jul 08 18:27:03.260734 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:03.260699 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:27:03.349612 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:03.349581 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:27:03.856389 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:03.856360 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks"] Jul 08 18:27:03.861223 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:03.861203 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:03.866013 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:03.865995 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-tcps8\"" Jul 08 18:27:03.868391 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:03.868372 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 18:27:03.891337 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:03.891315 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks"] Jul 08 18:27:03.894207 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:03.894188 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:27:04.001277 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.001247 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32ca5be3-1d43-41f4-9192-d06ff4660b5d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.001472 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.001294 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32ca5be3-1d43-41f4-9192-d06ff4660b5d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.001472 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.001319 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32ca5be3-1d43-41f4-9192-d06ff4660b5d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.001472 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.001386 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vcnx\" (UniqueName: \"kubernetes.io/projected/32ca5be3-1d43-41f4-9192-d06ff4660b5d-kube-api-access-6vcnx\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.001472 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.001463 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32ca5be3-1d43-41f4-9192-d06ff4660b5d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.001667 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.001507 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32ca5be3-1d43-41f4-9192-d06ff4660b5d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.001667 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.001538 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32ca5be3-1d43-41f4-9192-d06ff4660b5d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.001667 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.001561 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32ca5be3-1d43-41f4-9192-d06ff4660b5d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.001820 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.001668 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32ca5be3-1d43-41f4-9192-d06ff4660b5d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.093747 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.093715 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:27:04.102934 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.102907 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vcnx\" (UniqueName: \"kubernetes.io/projected/32ca5be3-1d43-41f4-9192-d06ff4660b5d-kube-api-access-6vcnx\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.103062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.102956 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32ca5be3-1d43-41f4-9192-d06ff4660b5d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.103062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.102983 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32ca5be3-1d43-41f4-9192-d06ff4660b5d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.103062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.103007 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32ca5be3-1d43-41f4-9192-d06ff4660b5d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.103062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.103030 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32ca5be3-1d43-41f4-9192-d06ff4660b5d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.103273 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.103105 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32ca5be3-1d43-41f4-9192-d06ff4660b5d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.103273 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.103167 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32ca5be3-1d43-41f4-9192-d06ff4660b5d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.103273 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.103199 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32ca5be3-1d43-41f4-9192-d06ff4660b5d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.103273 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.103229 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32ca5be3-1d43-41f4-9192-d06ff4660b5d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.103597 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.103567 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32ca5be3-1d43-41f4-9192-d06ff4660b5d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.104190 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.103890 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32ca5be3-1d43-41f4-9192-d06ff4660b5d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.104190 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.104030 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32ca5be3-1d43-41f4-9192-d06ff4660b5d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.104190 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.104027 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32ca5be3-1d43-41f4-9192-d06ff4660b5d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.104190 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.104041 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32ca5be3-1d43-41f4-9192-d06ff4660b5d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.105896 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.105832 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32ca5be3-1d43-41f4-9192-d06ff4660b5d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.108909 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.106293 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32ca5be3-1d43-41f4-9192-d06ff4660b5d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.114286 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.114227 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vcnx\" (UniqueName: \"kubernetes.io/projected/32ca5be3-1d43-41f4-9192-d06ff4660b5d-kube-api-access-6vcnx\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.115797 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.115708 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32ca5be3-1d43-41f4-9192-d06ff4660b5d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rdjks\" (UID: \"32ca5be3-1d43-41f4-9192-d06ff4660b5d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.170720 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.170692 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:04.317412 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.317371 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks"] Jul 08 18:27:04.323564 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:27:04.323533 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32ca5be3_1d43_41f4_9192_d06ff4660b5d.slice/crio-77cd497ba0d9ceb9379cb670fbcbc26d52d17ce0e6365ae71f7a2c7a140bf9ce WatchSource:0}: Error finding container 77cd497ba0d9ceb9379cb670fbcbc26d52d17ce0e6365ae71f7a2c7a140bf9ce: Status 404 returned error can't find the container with id 77cd497ba0d9ceb9379cb670fbcbc26d52d17ce0e6365ae71f7a2c7a140bf9ce Jul 08 18:27:04.513766 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.513651 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:27:04.646236 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.646202 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" event={"ID":"32ca5be3-1d43-41f4-9192-d06ff4660b5d","Type":"ContainerStarted","Data":"77cd497ba0d9ceb9379cb670fbcbc26d52d17ce0e6365ae71f7a2c7a140bf9ce"} Jul 08 18:27:04.718165 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.718135 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht"] Jul 08 18:27:04.722544 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.722526 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:04.725936 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.725916 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 08 18:27:04.726085 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.726058 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 08 18:27:04.726170 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.726153 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 08 18:27:04.726433 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.726418 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-pkdqr\"" Jul 08 18:27:04.742052 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.742031 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht"] Jul 08 18:27:04.910314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.910277 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gxhw\" (UniqueName: \"kubernetes.io/projected/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-kube-api-access-2gxhw\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:04.910475 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.910335 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:04.910475 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.910399 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:04.910475 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.910427 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:04.910475 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.910464 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:04.910657 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.910517 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:04.910657 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:04.910532 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:05.011497 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:05.011463 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:05.011497 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:05.011501 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:05.011746 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:05.011527 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gxhw\" (UniqueName: \"kubernetes.io/projected/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-kube-api-access-2gxhw\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:05.011746 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:05.011563 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:05.011746 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:05.011590 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:05.011746 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:05.011606 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:05.011746 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:05.011641 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:05.012549 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:05.012520 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:05.014078 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:05.014048 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:05.014181 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:05.014161 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:05.014228 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:05.014192 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:05.014263 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:05.014227 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:05.026908 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:05.026869 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:05.028461 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:05.028438 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gxhw\" (UniqueName: \"kubernetes.io/projected/cc6bb8b8-82ac-4b33-85ae-435ae7615f1f-kube-api-access-2gxhw\") pod \"istiod-openshift-gateway-94bb8fbfd-rkbht\" (UID: \"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:05.032263 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:05.032247 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:05.168306 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:05.168195 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht"] Jul 08 18:27:05.171498 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:27:05.171414 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc6bb8b8_82ac_4b33_85ae_435ae7615f1f.slice/crio-d1be6548e6a3d454674b844c3bc30c6f0c0b6f52947eda51232d873fa16ff5c2 WatchSource:0}: Error finding container d1be6548e6a3d454674b844c3bc30c6f0c0b6f52947eda51232d873fa16ff5c2: Status 404 returned error can't find the container with id d1be6548e6a3d454674b844c3bc30c6f0c0b6f52947eda51232d873fa16ff5c2 Jul 08 18:27:05.652779 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:05.652747 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" event={"ID":"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f","Type":"ContainerStarted","Data":"d1be6548e6a3d454674b844c3bc30c6f0c0b6f52947eda51232d873fa16ff5c2"} Jul 08 18:27:07.150613 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:07.150551 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:27:07.281172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:07.278693 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:27:07.281172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:07.279225 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:27:07.281172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:07.279386 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:27:07.281172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:07.279441 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:27:07.663785 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:07.663741 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" event={"ID":"32ca5be3-1d43-41f4-9192-d06ff4660b5d","Type":"ContainerStarted","Data":"842ab97e8150bb23d1dbe81329010a6754f6264e1d1cc904c8a5b3a484680561"} Jul 08 18:27:07.709773 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:07.709720 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" podStartSLOduration=1.756875343 podStartE2EDuration="4.709701616s" totalImagesPullingTime="2.952826273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:27:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:27:07.707562813 +0000 UTC m=+548.458872308" watchObservedRunningTime="2026-07-08 18:27:07.709701616 +0000 UTC m=+548.461011085" Jul 08 18:27:08.124492 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:08.124456 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:27:08.171744 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:08.171716 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:08.176335 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:08.176310 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:08.243322 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:08.243292 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:27:08.443250 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:08.443174 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:27:08.543220 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:08.543189 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:27:08.669595 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:08.669565 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:08.670677 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:08.670657 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rdjks" Jul 08 18:27:08.694468 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:08.694387 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:27:09.355921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:09.355888 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:27:09.356216 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:09.355952 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:27:09.675548 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:09.675459 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" event={"ID":"cc6bb8b8-82ac-4b33-85ae-435ae7615f1f","Type":"ContainerStarted","Data":"fab9defeacc2c8ba52d2a69fe34932d8b76be147152e5ba1eb401de70c227648"} Jul 08 18:27:09.699444 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:09.699385 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" podStartSLOduration=1.518823275 podStartE2EDuration="5.69936683s" totalImagesPullingTime="4.180543555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:27:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:27:09.697725709 +0000 UTC m=+550.449035186" watchObservedRunningTime="2026-07-08 18:27:09.69936683 +0000 UTC m=+550.450676305" Jul 08 18:27:10.630335 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:10.630304 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-56665c6f48-jc6wv" Jul 08 18:27:10.680013 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:10.679986 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:10.681736 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:10.681712 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rkbht" Jul 08 18:27:11.494353 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:11.494312 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:27:11.591747 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:11.591710 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:27:26.290330 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:26.290294 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:27:56.622243 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:56.622205 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:27:57.783038 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:27:57.783003 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:25.017811 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:25.017673 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:28.462330 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:28.462296 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:30.419792 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:30.419765 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:32.005394 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:32.005357 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:32.405379 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:32.405348 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:33.851245 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:33.851208 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:35.423493 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:35.423455 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:36.254666 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:36.254635 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:36.352014 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:36.351978 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:38.248143 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:38.248112 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:38.616374 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:38.616329 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:38.712036 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:38.712006 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:40.010330 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:40.010300 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:40.111568 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:40.111529 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:47.677041 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:47.677012 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:47.781750 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:47.781707 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:52.451741 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:52.451710 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:52.587471 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:52.587439 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:54.871068 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:54.871033 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:54.931016 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:28:54.930984 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:00.427998 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:00.427957 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:00.486695 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:00.486666 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:07.330841 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:07.330810 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:07.431176 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:07.431144 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:11.763156 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:11.763121 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:11.865135 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:11.865097 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:12.123590 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:12.123558 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:12.223298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:12.223236 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:14.580295 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:14.580262 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:14.681491 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:14.681456 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:18.267104 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:18.267075 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:18.530636 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:18.530554 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:20.532088 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:20.532048 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:20.632227 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:20.632194 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:23.118026 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:23.117986 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:23.235197 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:23.235163 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:23.306145 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:23.306105 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:23.979229 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:23.979200 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:24.077466 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:24.077437 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:27.739755 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:27.739726 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:27.840167 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:27.840124 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:32.538464 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:32.538430 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:32.638666 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:32.638628 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:34.990246 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:34.990215 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:35.091936 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:35.091899 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:37.146455 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:37.146418 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:37.199993 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:37.199964 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:44.433964 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:44.433930 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:44.533648 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:44.533619 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:44.884442 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:44.884400 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-snx52"] Jul 08 18:29:44.888004 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:44.887981 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-snx52" Jul 08 18:29:44.890598 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:44.890569 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 18:29:44.891283 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:44.891265 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 08 18:29:44.891426 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:44.891265 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 18:29:44.891517 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:44.891323 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-ljgz6\"" Jul 08 18:29:44.899310 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:44.899289 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-snx52"] Jul 08 18:29:44.925277 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:44.925231 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:45.032811 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:45.032776 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hmgg\" (UniqueName: \"kubernetes.io/projected/5e9883d2-2693-4607-8253-ce3d840e7fdb-kube-api-access-5hmgg\") pod \"postgres-69fb4594cf-snx52\" (UID: \"5e9883d2-2693-4607-8253-ce3d840e7fdb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-snx52" Jul 08 18:29:45.033003 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:45.032820 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5e9883d2-2693-4607-8253-ce3d840e7fdb-data\") pod \"postgres-69fb4594cf-snx52\" (UID: \"5e9883d2-2693-4607-8253-ce3d840e7fdb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-snx52" Jul 08 18:29:45.133654 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:45.133603 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hmgg\" (UniqueName: \"kubernetes.io/projected/5e9883d2-2693-4607-8253-ce3d840e7fdb-kube-api-access-5hmgg\") pod \"postgres-69fb4594cf-snx52\" (UID: \"5e9883d2-2693-4607-8253-ce3d840e7fdb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-snx52" Jul 08 18:29:45.133654 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:45.133666 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5e9883d2-2693-4607-8253-ce3d840e7fdb-data\") pod \"postgres-69fb4594cf-snx52\" (UID: \"5e9883d2-2693-4607-8253-ce3d840e7fdb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-snx52" Jul 08 18:29:45.134079 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:45.134058 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5e9883d2-2693-4607-8253-ce3d840e7fdb-data\") pod \"postgres-69fb4594cf-snx52\" (UID: \"5e9883d2-2693-4607-8253-ce3d840e7fdb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-snx52" Jul 08 18:29:45.143426 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:45.143368 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hmgg\" (UniqueName: \"kubernetes.io/projected/5e9883d2-2693-4607-8253-ce3d840e7fdb-kube-api-access-5hmgg\") pod \"postgres-69fb4594cf-snx52\" (UID: \"5e9883d2-2693-4607-8253-ce3d840e7fdb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-snx52" Jul 08 18:29:45.200519 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:45.200487 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-snx52" Jul 08 18:29:45.330320 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:45.330296 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-snx52"] Jul 08 18:29:45.332832 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:29:45.332796 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e9883d2_2693_4607_8253_ce3d840e7fdb.slice/crio-cb69e0839eabde69be0e37ef560253de072edbf093b8651c753efe792c0b65c6 WatchSource:0}: Error finding container cb69e0839eabde69be0e37ef560253de072edbf093b8651c753efe792c0b65c6: Status 404 returned error can't find the container with id cb69e0839eabde69be0e37ef560253de072edbf093b8651c753efe792c0b65c6 Jul 08 18:29:45.334008 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:45.333991 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:29:45.372698 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:45.372664 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-snx52" event={"ID":"5e9883d2-2693-4607-8253-ce3d840e7fdb","Type":"ContainerStarted","Data":"cb69e0839eabde69be0e37ef560253de072edbf093b8651c753efe792c0b65c6"} Jul 08 18:29:50.404145 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:50.404104 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-snx52" event={"ID":"5e9883d2-2693-4607-8253-ce3d840e7fdb","Type":"ContainerStarted","Data":"701e405ee05d944310712c74d3deb416f363b27b09a55e499742cb74f8a90e79"} Jul 08 18:29:50.404539 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:50.404234 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-snx52" Jul 08 18:29:50.423511 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:50.423451 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-snx52" podStartSLOduration=1.821765385 podStartE2EDuration="6.423433882s" totalImagesPullingTime="4.601668497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:29:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:29:50.420998867 +0000 UTC m=+711.172308343" watchObservedRunningTime="2026-07-08 18:29:50.423433882 +0000 UTC m=+711.174743357" Jul 08 18:29:54.853159 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:54.853125 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:56.439635 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:56.439604 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-snx52" Jul 08 18:29:56.770941 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:56.770858 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:56.871548 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:29:56.871522 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:03.828542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:30:03.828508 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:03.892344 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:30:03.892294 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:03.947752 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:30:03.947726 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:04.254703 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:30:04.254621 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:06.915546 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:30:06.915509 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:07.008915 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:30:07.008878 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:36.849814 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:30:36.849776 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:37.468217 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:30:37.468181 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:37.910168 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:30:37.910135 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:38.006306 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:30:38.006269 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:49.642547 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:30:49.642513 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:49.708151 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:30:49.708123 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:53.874120 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:30:53.874091 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:06.568253 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:06.568221 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:15.663496 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.663460 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w"] Jul 08 18:31:15.666339 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.666315 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.668814 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.668790 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 18:31:15.669913 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.669893 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rszwk\"" Jul 08 18:31:15.670027 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.669915 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 08 18:31:15.670027 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.669893 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 18:31:15.677501 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.677478 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w"] Jul 08 18:31:15.699370 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.699340 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:15.759633 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.759595 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.759806 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.759670 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.759806 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.759742 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.759806 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.759782 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.759806 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.759807 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.760010 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.759862 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccbs6\" (UniqueName: \"kubernetes.io/projected/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-kube-api-access-ccbs6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.860581 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.860551 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.860797 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.860611 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.860797 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.860657 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.860797 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.860682 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.860797 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.860711 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.860797 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.860737 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccbs6\" (UniqueName: \"kubernetes.io/projected/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-kube-api-access-ccbs6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.861105 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.861074 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.861177 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.861151 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.861215 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.861170 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.863147 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.863119 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.863425 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.863406 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.868896 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.868874 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccbs6\" (UniqueName: \"kubernetes.io/projected/e1eb580e-486c-40e6-8ddf-a34ae825e3b6-kube-api-access-ccbs6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds24w\" (UID: \"e1eb580e-486c-40e6-8ddf-a34ae825e3b6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:15.979410 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:15.979327 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:16.117264 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:16.117237 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w"] Jul 08 18:31:16.118558 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:31:16.118534 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1eb580e_486c_40e6_8ddf_a34ae825e3b6.slice/crio-b2f7309261aba6bd2da1dd8389c21a2f1fabbdf15e9213ac53849121de3ceb90 WatchSource:0}: Error finding container b2f7309261aba6bd2da1dd8389c21a2f1fabbdf15e9213ac53849121de3ceb90: Status 404 returned error can't find the container with id b2f7309261aba6bd2da1dd8389c21a2f1fabbdf15e9213ac53849121de3ceb90 Jul 08 18:31:16.779783 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:16.779752 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" event={"ID":"e1eb580e-486c-40e6-8ddf-a34ae825e3b6","Type":"ContainerStarted","Data":"b2f7309261aba6bd2da1dd8389c21a2f1fabbdf15e9213ac53849121de3ceb90"} Jul 08 18:31:17.784939 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:17.784901 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" event={"ID":"e1eb580e-486c-40e6-8ddf-a34ae825e3b6","Type":"ContainerStarted","Data":"6fdc9348f41880c6d4e959240a669fe2d8b49b76f8017f9e417836af5eaebf16"} Jul 08 18:31:17.802910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:17.802861 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" podStartSLOduration=1.814473309 podStartE2EDuration="2.802831264s" totalImagesPullingTime="988.357955ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:31:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:31:17.802166196 +0000 UTC m=+798.553475672" watchObservedRunningTime="2026-07-08 18:31:17.802831264 +0000 UTC m=+798.554140738" Jul 08 18:31:25.980337 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:25.980245 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:25.980337 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:25.980313 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:25.992544 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:25.992519 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:26.834037 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:26.834006 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds24w" Jul 08 18:31:30.168245 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.168217 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw"] Jul 08 18:31:30.170829 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.170808 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.173277 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.173255 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 08 18:31:30.181988 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.181966 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw"] Jul 08 18:31:30.194288 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.194260 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:30.287136 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.287098 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43b85d6e-5201-4d92-bba3-b4df7346b1e2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.287337 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.287145 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74dgp\" (UniqueName: \"kubernetes.io/projected/43b85d6e-5201-4d92-bba3-b4df7346b1e2-kube-api-access-74dgp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.287337 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.287217 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43b85d6e-5201-4d92-bba3-b4df7346b1e2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.287337 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.287264 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43b85d6e-5201-4d92-bba3-b4df7346b1e2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.287337 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.287293 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43b85d6e-5201-4d92-bba3-b4df7346b1e2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.287337 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.287310 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43b85d6e-5201-4d92-bba3-b4df7346b1e2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.388639 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.388602 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74dgp\" (UniqueName: \"kubernetes.io/projected/43b85d6e-5201-4d92-bba3-b4df7346b1e2-kube-api-access-74dgp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.388835 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.388651 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43b85d6e-5201-4d92-bba3-b4df7346b1e2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.388835 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.388673 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43b85d6e-5201-4d92-bba3-b4df7346b1e2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.388835 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.388695 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43b85d6e-5201-4d92-bba3-b4df7346b1e2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.388835 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.388721 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43b85d6e-5201-4d92-bba3-b4df7346b1e2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.388835 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.388761 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43b85d6e-5201-4d92-bba3-b4df7346b1e2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.389152 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.389113 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43b85d6e-5201-4d92-bba3-b4df7346b1e2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.389202 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.389145 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43b85d6e-5201-4d92-bba3-b4df7346b1e2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.389202 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.389186 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43b85d6e-5201-4d92-bba3-b4df7346b1e2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.391135 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.391114 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43b85d6e-5201-4d92-bba3-b4df7346b1e2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.391584 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.391564 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43b85d6e-5201-4d92-bba3-b4df7346b1e2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.397051 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.397029 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74dgp\" (UniqueName: \"kubernetes.io/projected/43b85d6e-5201-4d92-bba3-b4df7346b1e2-kube-api-access-74dgp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s7hhw\" (UID: \"43b85d6e-5201-4d92-bba3-b4df7346b1e2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.482073 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.481980 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:30.612754 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.612728 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw"] Jul 08 18:31:30.614933 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:31:30.614904 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43b85d6e_5201_4d92_bba3_b4df7346b1e2.slice/crio-adb5bcc72b72853d67169757fa1018144763e0c261808fc5ee6c8eecc21a8b5a WatchSource:0}: Error finding container adb5bcc72b72853d67169757fa1018144763e0c261808fc5ee6c8eecc21a8b5a: Status 404 returned error can't find the container with id adb5bcc72b72853d67169757fa1018144763e0c261808fc5ee6c8eecc21a8b5a Jul 08 18:31:30.840305 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:30.840277 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" event={"ID":"43b85d6e-5201-4d92-bba3-b4df7346b1e2","Type":"ContainerStarted","Data":"adb5bcc72b72853d67169757fa1018144763e0c261808fc5ee6c8eecc21a8b5a"} Jul 08 18:31:31.845744 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:31.845707 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" event={"ID":"43b85d6e-5201-4d92-bba3-b4df7346b1e2","Type":"ContainerStarted","Data":"504ff2b48afeab52d3154a3468d291f1523a0309146eb5a787fc51c4f4289946"} Jul 08 18:31:31.862799 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:31.862749 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" podStartSLOduration=1.668030159 podStartE2EDuration="1.862734326s" totalImagesPullingTime="194.704167ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:31:31.861250023 +0000 UTC m=+812.612559501" watchObservedRunningTime="2026-07-08 18:31:31.862734326 +0000 UTC m=+812.614043799" Jul 08 18:31:35.402246 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:35.402212 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:40.482942 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:40.482888 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:40.483331 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:40.482968 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:40.495514 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:40.495493 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:31:40.897949 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:31:40.897901 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s7hhw" Jul 08 18:32:04.909693 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:32:04.909632 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:19.008079 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:32:19.008047 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:48.941988 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:32:48.941955 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:33:09.923885 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:33:09.923834 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:33:11.308357 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:33:11.308325 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:33:34.524047 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:33:34.524011 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:33:34.621326 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:33:34.621288 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:33:37.575089 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:33:37.575055 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:33:37.634036 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:33:37.634003 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:22.522353 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:34:22.522266 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:24.574806 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:34:24.574771 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:47.812361 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:34:47.812328 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:05.431784 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:35:05.431727 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:05.531111 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:35:05.531080 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:05.702373 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:35:05.702291 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:11.358689 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:35:11.358646 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:11.463648 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:35:11.463603 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:11.579366 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:35:11.579330 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:11.750214 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:35:11.750116 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:17.685317 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:35:17.685282 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:17.783765 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:35:17.783718 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:36:01.203130 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:36:01.203031 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:36:01.303768 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:36:01.303733 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:36:41.424730 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:36:41.424696 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:39:05.905494 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:39:05.905407 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:39:07.409037 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:39:07.409003 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:39:10.803589 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:39:10.803555 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:40:48.358389 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:40:48.358352 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:40:48.450160 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:40:48.450129 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:41:19.569802 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:41:19.569762 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:44:46.399898 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:44:46.399862 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:06.438058 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:06.438021 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:16.047534 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:16.047491 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:24.306771 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:24.306694 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:29.478914 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:29.478882 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:32.500184 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:32.500154 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:32.600564 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:32.600528 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:32.989171 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:32.989136 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:38.739182 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:38.739153 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:38.802000 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:38.801972 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:38.937830 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:38.937795 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl"] Jul 08 18:46:38.943078 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:38.943048 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:38.945722 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:38.945699 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-40bf37b7-openshift-default-dockercfg-lhdnp\"" Jul 08 18:46:38.957219 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:38.957197 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl"] Jul 08 18:46:38.975492 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:38.975467 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:39.070397 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.070369 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:39.082084 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.082060 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-token\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.082195 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.082100 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-workload-certs\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.082195 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.082119 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-workload-socket\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.082195 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.082134 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-data\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.082195 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.082176 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-credential-socket\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.082329 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.082214 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qxgn\" (UniqueName: \"kubernetes.io/projected/6f772584-c374-4195-b57a-3a672c6fbb4e-kube-api-access-4qxgn\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.082329 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.082246 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-envoy\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.082329 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.082269 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f772584-c374-4195-b57a-3a672c6fbb4e-istiod-ca-cert\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.082420 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.082345 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-podinfo\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.183508 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.183471 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qxgn\" (UniqueName: \"kubernetes.io/projected/6f772584-c374-4195-b57a-3a672c6fbb4e-kube-api-access-4qxgn\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.183660 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.183524 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-envoy\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.183720 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.183697 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f772584-c374-4195-b57a-3a672c6fbb4e-istiod-ca-cert\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.183777 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.183759 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-podinfo\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.183886 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.183837 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-token\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.183958 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.183912 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-workload-certs\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.183958 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.183940 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-workload-socket\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.184061 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.183963 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-data\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.184061 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.183999 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-credential-socket\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.184277 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.184246 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-workload-certs\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.184407 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.184338 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-workload-socket\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.184475 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.184439 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-data\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.184589 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.184573 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f772584-c374-4195-b57a-3a672c6fbb4e-istiod-ca-cert\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.184633 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.184592 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-credential-socket\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.185942 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.185920 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-envoy\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.186274 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.186258 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-podinfo\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.194288 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.194263 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-token\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.194403 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.194331 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qxgn\" (UniqueName: \"kubernetes.io/projected/6f772584-c374-4195-b57a-3a672c6fbb4e-kube-api-access-4qxgn\") pod \"e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.253530 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.253488 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:39.388125 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.388101 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl"] Jul 08 18:46:39.390255 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:46:39.390228 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f772584_c374_4195_b57a_3a672c6fbb4e.slice/crio-63832ded78d3d475edba016382a2b13d5fb97d72688da6eb6fa8adea35229d39 WatchSource:0}: Error finding container 63832ded78d3d475edba016382a2b13d5fb97d72688da6eb6fa8adea35229d39: Status 404 returned error can't find the container with id 63832ded78d3d475edba016382a2b13d5fb97d72688da6eb6fa8adea35229d39 Jul 08 18:46:39.392275 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.392244 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:46:39.392333 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.392309 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:46:39.392373 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.392340 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:46:39.899053 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.899019 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" event={"ID":"6f772584-c374-4195-b57a-3a672c6fbb4e","Type":"ContainerStarted","Data":"d3979746f84c17b9ec3631ea173fa30f7831eb422b7bfa917db92fb44945cfcf"} Jul 08 18:46:39.899491 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.899073 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" event={"ID":"6f772584-c374-4195-b57a-3a672c6fbb4e","Type":"ContainerStarted","Data":"63832ded78d3d475edba016382a2b13d5fb97d72688da6eb6fa8adea35229d39"} Jul 08 18:46:39.921198 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:39.921149 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" podStartSLOduration=1.921134422 podStartE2EDuration="1.921134422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:46:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:46:39.919792884 +0000 UTC m=+1720.671102360" watchObservedRunningTime="2026-07-08 18:46:39.921134422 +0000 UTC m=+1720.672443895" Jul 08 18:46:40.049725 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:40.049689 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:40.190011 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:40.189926 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:40.253893 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:40.253833 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:40.258915 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:40.258888 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:40.288089 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:40.288059 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:40.745673 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:40.745638 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:40.850371 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:40.850344 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:40.905056 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:40.905032 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:40.905960 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:40.905940 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:42.800869 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:42.800826 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:45.517678 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:45.517641 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:45.714491 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:45.714458 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl"] Jul 08 18:46:45.714703 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:45.714678 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" podUID="6f772584-c374-4195-b57a-3a672c6fbb4e" containerName="istio-proxy" containerID="cri-o://d3979746f84c17b9ec3631ea173fa30f7831eb422b7bfa917db92fb44945cfcf" gracePeriod=30 Jul 08 18:46:45.738444 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:45.738408 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:45.849215 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:45.849186 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:45.911094 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:45.911060 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:47.148224 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:47.148196 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:51.591907 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:51.591866 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:51.692432 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:51.692398 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:51.907520 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:51.907479 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:51.957242 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:51.957216 2574 generic.go:356] "Generic (PLEG): container finished" podID="6f772584-c374-4195-b57a-3a672c6fbb4e" containerID="d3979746f84c17b9ec3631ea173fa30f7831eb422b7bfa917db92fb44945cfcf" exitCode=0 Jul 08 18:46:51.957385 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:51.957297 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" event={"ID":"6f772584-c374-4195-b57a-3a672c6fbb4e","Type":"ContainerDied","Data":"d3979746f84c17b9ec3631ea173fa30f7831eb422b7bfa917db92fb44945cfcf"} Jul 08 18:46:51.957385 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:51.957358 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" event={"ID":"6f772584-c374-4195-b57a-3a672c6fbb4e","Type":"ContainerDied","Data":"63832ded78d3d475edba016382a2b13d5fb97d72688da6eb6fa8adea35229d39"} Jul 08 18:46:51.957385 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:51.957381 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63832ded78d3d475edba016382a2b13d5fb97d72688da6eb6fa8adea35229d39" Jul 08 18:46:51.964088 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:51.964068 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:52.006936 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.006891 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:52.011091 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.011071 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f772584-c374-4195-b57a-3a672c6fbb4e-istiod-ca-cert\") pod \"6f772584-c374-4195-b57a-3a672c6fbb4e\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " Jul 08 18:46:52.011203 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.011132 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qxgn\" (UniqueName: \"kubernetes.io/projected/6f772584-c374-4195-b57a-3a672c6fbb4e-kube-api-access-4qxgn\") pod \"6f772584-c374-4195-b57a-3a672c6fbb4e\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " Jul 08 18:46:52.011203 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.011170 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-data\") pod \"6f772584-c374-4195-b57a-3a672c6fbb4e\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " Jul 08 18:46:52.011203 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.011200 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-podinfo\") pod \"6f772584-c374-4195-b57a-3a672c6fbb4e\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " Jul 08 18:46:52.011352 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.011218 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-token\") pod \"6f772584-c374-4195-b57a-3a672c6fbb4e\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " Jul 08 18:46:52.011352 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.011252 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-envoy\") pod \"6f772584-c374-4195-b57a-3a672c6fbb4e\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " Jul 08 18:46:52.011352 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.011288 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-credential-socket\") pod \"6f772584-c374-4195-b57a-3a672c6fbb4e\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " Jul 08 18:46:52.011352 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.011313 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-workload-certs\") pod \"6f772584-c374-4195-b57a-3a672c6fbb4e\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " Jul 08 18:46:52.011352 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.011339 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-workload-socket\") pod \"6f772584-c374-4195-b57a-3a672c6fbb4e\" (UID: \"6f772584-c374-4195-b57a-3a672c6fbb4e\") " Jul 08 18:46:52.011636 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.011573 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f772584-c374-4195-b57a-3a672c6fbb4e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6f772584-c374-4195-b57a-3a672c6fbb4e" (UID: "6f772584-c374-4195-b57a-3a672c6fbb4e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:46:52.011804 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.011779 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6f772584-c374-4195-b57a-3a672c6fbb4e" (UID: "6f772584-c374-4195-b57a-3a672c6fbb4e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:46:52.011916 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.011894 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6f772584-c374-4195-b57a-3a672c6fbb4e" (UID: "6f772584-c374-4195-b57a-3a672c6fbb4e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:46:52.012029 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.011913 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6f772584-c374-4195-b57a-3a672c6fbb4e" (UID: "6f772584-c374-4195-b57a-3a672c6fbb4e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:46:52.012197 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.012115 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-data" (OuterVolumeSpecName: "istio-data") pod "6f772584-c374-4195-b57a-3a672c6fbb4e" (UID: "6f772584-c374-4195-b57a-3a672c6fbb4e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:46:52.013659 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.013631 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f772584-c374-4195-b57a-3a672c6fbb4e-kube-api-access-4qxgn" (OuterVolumeSpecName: "kube-api-access-4qxgn") pod "6f772584-c374-4195-b57a-3a672c6fbb4e" (UID: "6f772584-c374-4195-b57a-3a672c6fbb4e"). InnerVolumeSpecName "kube-api-access-4qxgn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:46:52.013808 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.013786 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-token" (OuterVolumeSpecName: "istio-token") pod "6f772584-c374-4195-b57a-3a672c6fbb4e" (UID: "6f772584-c374-4195-b57a-3a672c6fbb4e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:46:52.013939 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.013815 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6f772584-c374-4195-b57a-3a672c6fbb4e" (UID: "6f772584-c374-4195-b57a-3a672c6fbb4e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:46:52.014052 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.014030 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6f772584-c374-4195-b57a-3a672c6fbb4e" (UID: "6f772584-c374-4195-b57a-3a672c6fbb4e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:46:52.113057 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.113017 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-podinfo\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:46:52.113057 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.113050 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-token\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:46:52.113057 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.113063 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-envoy\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:46:52.113295 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.113074 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-credential-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:46:52.113295 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.113086 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-workload-certs\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:46:52.113295 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.113098 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-workload-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:46:52.113295 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.113108 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f772584-c374-4195-b57a-3a672c6fbb4e-istiod-ca-cert\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:46:52.113295 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.113119 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qxgn\" (UniqueName: \"kubernetes.io/projected/6f772584-c374-4195-b57a-3a672c6fbb4e-kube-api-access-4qxgn\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:46:52.113295 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.113130 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f772584-c374-4195-b57a-3a672c6fbb4e-istio-data\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:46:52.150483 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.150409 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:52.961288 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.961250 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:52.962293 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.962270 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl" Jul 08 18:46:52.992588 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.992562 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl"] Jul 08 18:46:52.999540 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:52.999522 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-40bf37b7-openshift-default-649bc55b85-d6xfl"] Jul 08 18:46:53.047354 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.047322 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x"] Jul 08 18:46:53.047744 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.047727 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f772584-c374-4195-b57a-3a672c6fbb4e" containerName="istio-proxy" Jul 08 18:46:53.047831 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.047748 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f772584-c374-4195-b57a-3a672c6fbb4e" containerName="istio-proxy" Jul 08 18:46:53.047909 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.047841 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f772584-c374-4195-b57a-3a672c6fbb4e" containerName="istio-proxy" Jul 08 18:46:53.050205 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.050187 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" Jul 08 18:46:53.052811 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.052793 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rlgwv\"" Jul 08 18:46:53.052929 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.052834 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-68a2310a-serving-cert\"" Jul 08 18:46:53.062155 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.062131 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x"] Jul 08 18:46:53.074807 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.074777 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:53.120620 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.120586 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-maas-api-tls\") pod \"maas-api-e2e-ait-68a2310a-58949546c6-kvz4x\" (UID: \"44d0718a-138c-4fa0-b2a5-f0a5d737a88f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" Jul 08 18:46:53.120779 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.120700 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-openshift-service-ca\") pod \"maas-api-e2e-ait-68a2310a-58949546c6-kvz4x\" (UID: \"44d0718a-138c-4fa0-b2a5-f0a5d737a88f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" Jul 08 18:46:53.120779 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.120734 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrpbt\" (UniqueName: \"kubernetes.io/projected/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-kube-api-access-mrpbt\") pod \"maas-api-e2e-ait-68a2310a-58949546c6-kvz4x\" (UID: \"44d0718a-138c-4fa0-b2a5-f0a5d737a88f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" Jul 08 18:46:53.148233 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.148205 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:53.222325 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.222233 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-openshift-service-ca\") pod \"maas-api-e2e-ait-68a2310a-58949546c6-kvz4x\" (UID: \"44d0718a-138c-4fa0-b2a5-f0a5d737a88f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" Jul 08 18:46:53.222325 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.222277 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrpbt\" (UniqueName: \"kubernetes.io/projected/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-kube-api-access-mrpbt\") pod \"maas-api-e2e-ait-68a2310a-58949546c6-kvz4x\" (UID: \"44d0718a-138c-4fa0-b2a5-f0a5d737a88f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" Jul 08 18:46:53.222325 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.222324 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-maas-api-tls\") pod \"maas-api-e2e-ait-68a2310a-58949546c6-kvz4x\" (UID: \"44d0718a-138c-4fa0-b2a5-f0a5d737a88f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" Jul 08 18:46:53.223107 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.223078 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-openshift-service-ca\") pod \"maas-api-e2e-ait-68a2310a-58949546c6-kvz4x\" (UID: \"44d0718a-138c-4fa0-b2a5-f0a5d737a88f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" Jul 08 18:46:53.225266 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.225243 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-maas-api-tls\") pod \"maas-api-e2e-ait-68a2310a-58949546c6-kvz4x\" (UID: \"44d0718a-138c-4fa0-b2a5-f0a5d737a88f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" Jul 08 18:46:53.234138 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.234114 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrpbt\" (UniqueName: \"kubernetes.io/projected/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-kube-api-access-mrpbt\") pod \"maas-api-e2e-ait-68a2310a-58949546c6-kvz4x\" (UID: \"44d0718a-138c-4fa0-b2a5-f0a5d737a88f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" Jul 08 18:46:53.243999 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.243979 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:53.360642 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.360609 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" Jul 08 18:46:53.492067 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.492040 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x"] Jul 08 18:46:53.494960 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:46:53.494922 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44d0718a_138c_4fa0_b2a5_f0a5d737a88f.slice/crio-7114bf756c723bfba3bc9ece1d281d293f7645936a713a13eedaad5e230fa7f3 WatchSource:0}: Error finding container 7114bf756c723bfba3bc9ece1d281d293f7645936a713a13eedaad5e230fa7f3: Status 404 returned error can't find the container with id 7114bf756c723bfba3bc9ece1d281d293f7645936a713a13eedaad5e230fa7f3 Jul 08 18:46:53.496479 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.496459 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:46:53.629577 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.629545 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:53.735348 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.735317 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:53.773689 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.773604 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f772584-c374-4195-b57a-3a672c6fbb4e" path="/var/lib/kubelet/pods/6f772584-c374-4195-b57a-3a672c6fbb4e/volumes" Jul 08 18:46:53.968166 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:53.968130 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" event={"ID":"44d0718a-138c-4fa0-b2a5-f0a5d737a88f","Type":"ContainerStarted","Data":"7114bf756c723bfba3bc9ece1d281d293f7645936a713a13eedaad5e230fa7f3"} Jul 08 18:46:55.342552 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:55.342088 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:55.979304 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:55.979219 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" event={"ID":"44d0718a-138c-4fa0-b2a5-f0a5d737a88f","Type":"ContainerStarted","Data":"94400f7297c6c6590e76832acc3f1d8f04deb520811512e81358e7120266cd34"} Jul 08 18:46:55.979460 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:55.979367 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" Jul 08 18:46:55.996651 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:55.996604 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" podStartSLOduration=0.795672287 podStartE2EDuration="2.996590068s" totalImagesPullingTime="2.200917781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:46:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:46:55.995243332 +0000 UTC m=+1736.746552808" watchObservedRunningTime="2026-07-08 18:46:55.996590068 +0000 UTC m=+1736.747899542" Jul 08 18:46:58.118033 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:58.117980 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:58.422999 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:58.422926 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x"] Jul 08 18:46:58.423130 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:58.423104 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" podUID="44d0718a-138c-4fa0-b2a5-f0a5d737a88f" containerName="maas-api" containerID="cri-o://94400f7297c6c6590e76832acc3f1d8f04deb520811512e81358e7120266cd34" gracePeriod=30 Jul 08 18:46:58.451538 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:58.451513 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:46:58.663480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:58.663453 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" Jul 08 18:46:58.777461 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:58.777377 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrpbt\" (UniqueName: \"kubernetes.io/projected/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-kube-api-access-mrpbt\") pod \"44d0718a-138c-4fa0-b2a5-f0a5d737a88f\" (UID: \"44d0718a-138c-4fa0-b2a5-f0a5d737a88f\") " Jul 08 18:46:58.777461 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:58.777434 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-openshift-service-ca\") pod \"44d0718a-138c-4fa0-b2a5-f0a5d737a88f\" (UID: \"44d0718a-138c-4fa0-b2a5-f0a5d737a88f\") " Jul 08 18:46:58.777692 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:58.777481 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-maas-api-tls\") pod \"44d0718a-138c-4fa0-b2a5-f0a5d737a88f\" (UID: \"44d0718a-138c-4fa0-b2a5-f0a5d737a88f\") " Jul 08 18:46:58.777833 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:58.777808 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "44d0718a-138c-4fa0-b2a5-f0a5d737a88f" (UID: "44d0718a-138c-4fa0-b2a5-f0a5d737a88f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:46:58.779807 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:58.779785 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-kube-api-access-mrpbt" (OuterVolumeSpecName: "kube-api-access-mrpbt") pod "44d0718a-138c-4fa0-b2a5-f0a5d737a88f" (UID: "44d0718a-138c-4fa0-b2a5-f0a5d737a88f"). InnerVolumeSpecName "kube-api-access-mrpbt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:46:58.779923 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:58.779808 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "44d0718a-138c-4fa0-b2a5-f0a5d737a88f" (UID: "44d0718a-138c-4fa0-b2a5-f0a5d737a88f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:46:58.878499 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:58.878469 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrpbt\" (UniqueName: \"kubernetes.io/projected/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-kube-api-access-mrpbt\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:46:58.878499 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:58.878498 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-openshift-service-ca\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:46:58.878692 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:58.878512 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d0718a-138c-4fa0-b2a5-f0a5d737a88f-maas-api-tls\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:46:59.000894 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:59.000838 2574 generic.go:356] "Generic (PLEG): container finished" podID="44d0718a-138c-4fa0-b2a5-f0a5d737a88f" containerID="94400f7297c6c6590e76832acc3f1d8f04deb520811512e81358e7120266cd34" exitCode=0 Jul 08 18:46:59.000894 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:59.000886 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" event={"ID":"44d0718a-138c-4fa0-b2a5-f0a5d737a88f","Type":"ContainerDied","Data":"94400f7297c6c6590e76832acc3f1d8f04deb520811512e81358e7120266cd34"} Jul 08 18:46:59.001109 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:59.000918 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" Jul 08 18:46:59.001109 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:59.000940 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x" event={"ID":"44d0718a-138c-4fa0-b2a5-f0a5d737a88f","Type":"ContainerDied","Data":"7114bf756c723bfba3bc9ece1d281d293f7645936a713a13eedaad5e230fa7f3"} Jul 08 18:46:59.001109 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:59.000964 2574 scope.go:117] "RemoveContainer" containerID="94400f7297c6c6590e76832acc3f1d8f04deb520811512e81358e7120266cd34" Jul 08 18:46:59.009745 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:59.009730 2574 scope.go:117] "RemoveContainer" containerID="94400f7297c6c6590e76832acc3f1d8f04deb520811512e81358e7120266cd34" Jul 08 18:46:59.010029 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:46:59.010007 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94400f7297c6c6590e76832acc3f1d8f04deb520811512e81358e7120266cd34\": container with ID starting with 94400f7297c6c6590e76832acc3f1d8f04deb520811512e81358e7120266cd34 not found: ID does not exist" containerID="94400f7297c6c6590e76832acc3f1d8f04deb520811512e81358e7120266cd34" Jul 08 18:46:59.010113 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:59.010035 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94400f7297c6c6590e76832acc3f1d8f04deb520811512e81358e7120266cd34"} err="failed to get container status \"94400f7297c6c6590e76832acc3f1d8f04deb520811512e81358e7120266cd34\": rpc error: code = NotFound desc = could not find container \"94400f7297c6c6590e76832acc3f1d8f04deb520811512e81358e7120266cd34\": container with ID starting with 94400f7297c6c6590e76832acc3f1d8f04deb520811512e81358e7120266cd34 not found: ID does not exist" Jul 08 18:46:59.020927 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:59.020907 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x"] Jul 08 18:46:59.026809 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:59.026786 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-68a2310a-58949546c6-kvz4x"] Jul 08 18:46:59.776811 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:46:59.772347 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44d0718a-138c-4fa0-b2a5-f0a5d737a88f" path="/var/lib/kubelet/pods/44d0718a-138c-4fa0-b2a5-f0a5d737a88f/volumes" Jul 08 18:47:04.253991 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:04.253958 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:04.380740 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:04.380706 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:06.629137 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:06.629100 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:10.252766 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.252727 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:10.350802 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.350764 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:10.435112 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.435078 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf"] Jul 08 18:47:10.435583 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.435562 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44d0718a-138c-4fa0-b2a5-f0a5d737a88f" containerName="maas-api" Jul 08 18:47:10.435583 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.435585 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="44d0718a-138c-4fa0-b2a5-f0a5d737a88f" containerName="maas-api" Jul 08 18:47:10.435765 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.435662 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="44d0718a-138c-4fa0-b2a5-f0a5d737a88f" containerName="maas-api" Jul 08 18:47:10.440536 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.440515 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.443822 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.443800 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-8df77702-openshift-default-dockercfg-zck2q\"" Jul 08 18:47:10.456296 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.456270 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf"] Jul 08 18:47:10.478533 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.478500 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:10.543638 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.543547 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:10.586548 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.586511 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12521138-faa2-4657-99f5-a2c24efbd754-istio-podinfo\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.586713 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.586554 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-workload-socket\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.586713 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.586576 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-workload-certs\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.586713 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.586662 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12521138-faa2-4657-99f5-a2c24efbd754-istio-token\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.586821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.586717 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-credential-socket\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.586821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.586745 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-istio-data\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.586821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.586791 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12521138-faa2-4657-99f5-a2c24efbd754-istiod-ca-cert\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.586952 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.586824 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd4lb\" (UniqueName: \"kubernetes.io/projected/12521138-faa2-4657-99f5-a2c24efbd754-kube-api-access-hd4lb\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.586952 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.586933 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-istio-envoy\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.687496 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.687461 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12521138-faa2-4657-99f5-a2c24efbd754-istiod-ca-cert\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.687677 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.687504 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd4lb\" (UniqueName: \"kubernetes.io/projected/12521138-faa2-4657-99f5-a2c24efbd754-kube-api-access-hd4lb\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.687677 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.687585 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-istio-envoy\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.687677 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.687614 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12521138-faa2-4657-99f5-a2c24efbd754-istio-podinfo\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.687677 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.687652 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-workload-socket\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.687677 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.687674 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-workload-certs\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.687986 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.687698 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12521138-faa2-4657-99f5-a2c24efbd754-istio-token\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.687986 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.687739 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-credential-socket\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.687986 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.687774 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-istio-data\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.688137 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.688107 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-workload-socket\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.688266 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.688244 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12521138-faa2-4657-99f5-a2c24efbd754-istiod-ca-cert\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.688362 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.688342 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-istio-data\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.688514 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.688395 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-credential-socket\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.688514 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.688417 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-workload-certs\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.690089 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.690066 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-istio-envoy\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.690306 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.690290 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12521138-faa2-4657-99f5-a2c24efbd754-istio-podinfo\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.695571 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.695545 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12521138-faa2-4657-99f5-a2c24efbd754-istio-token\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.695735 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.695716 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd4lb\" (UniqueName: \"kubernetes.io/projected/12521138-faa2-4657-99f5-a2c24efbd754-kube-api-access-hd4lb\") pod \"e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.752438 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.752407 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:10.890608 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.890485 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf"] Jul 08 18:47:10.892878 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:47:10.892816 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12521138_faa2_4657_99f5_a2c24efbd754.slice/crio-a6e492f2d120da7e3b790e903e806f1dc5b5fd9ee4c15623e209fd2e5e395964 WatchSource:0}: Error finding container a6e492f2d120da7e3b790e903e806f1dc5b5fd9ee4c15623e209fd2e5e395964: Status 404 returned error can't find the container with id a6e492f2d120da7e3b790e903e806f1dc5b5fd9ee4c15623e209fd2e5e395964 Jul 08 18:47:10.894932 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.894899 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:47:10.895047 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.894972 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:47:10.895047 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:10.894999 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:47:11.057375 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.057298 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" event={"ID":"12521138-faa2-4657-99f5-a2c24efbd754","Type":"ContainerStarted","Data":"50344ceebc68f174820e431e13f2c5b221a90a1c4a0929fd5ba286f39d9c3a5a"} Jul 08 18:47:11.057375 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.057351 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" event={"ID":"12521138-faa2-4657-99f5-a2c24efbd754","Type":"ContainerStarted","Data":"a6e492f2d120da7e3b790e903e806f1dc5b5fd9ee4c15623e209fd2e5e395964"} Jul 08 18:47:11.079341 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.079283 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" podStartSLOduration=1.079261157 podStartE2EDuration="1.079261157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:47:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:47:11.077510015 +0000 UTC m=+1751.828819484" watchObservedRunningTime="2026-07-08 18:47:11.079261157 +0000 UTC m=+1751.830570632" Jul 08 18:47:11.570181 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.570147 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:11.651253 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.651222 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln"] Jul 08 18:47:11.654455 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.654434 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:11.654946 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.654929 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" Jul 08 18:47:11.657359 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.657341 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-8df77702-serving-cert\"" Jul 08 18:47:11.657572 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.657558 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rlgwv\"" Jul 08 18:47:11.668501 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.668479 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln"] Jul 08 18:47:11.722297 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.722268 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:11.752880 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.752826 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:11.780415 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.780368 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:11.798944 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.798918 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgnbw\" (UniqueName: \"kubernetes.io/projected/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-kube-api-access-dgnbw\") pod \"maas-api-e2e-derive-8df77702-667bbf544d-h2hln\" (UID: \"3e2abd37-be9f-4cad-bb82-12e6f8f2768f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" Jul 08 18:47:11.799084 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.798998 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-openshift-service-ca\") pod \"maas-api-e2e-derive-8df77702-667bbf544d-h2hln\" (UID: \"3e2abd37-be9f-4cad-bb82-12e6f8f2768f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" Jul 08 18:47:11.799084 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.799055 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-maas-api-tls\") pod \"maas-api-e2e-derive-8df77702-667bbf544d-h2hln\" (UID: \"3e2abd37-be9f-4cad-bb82-12e6f8f2768f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" Jul 08 18:47:11.900247 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.900147 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-maas-api-tls\") pod \"maas-api-e2e-derive-8df77702-667bbf544d-h2hln\" (UID: \"3e2abd37-be9f-4cad-bb82-12e6f8f2768f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" Jul 08 18:47:11.900419 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.900253 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgnbw\" (UniqueName: \"kubernetes.io/projected/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-kube-api-access-dgnbw\") pod \"maas-api-e2e-derive-8df77702-667bbf544d-h2hln\" (UID: \"3e2abd37-be9f-4cad-bb82-12e6f8f2768f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" Jul 08 18:47:11.900419 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:47:11.900315 2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-serving-cert: secret "maas-api-e2e-derive-8df77702-serving-cert" not found Jul 08 18:47:11.900419 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:47:11.900389 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-maas-api-tls podName:3e2abd37-be9f-4cad-bb82-12e6f8f2768f nodeName:}" failed. No retries permitted until 2026-07-08 18:47:12.400367154 +0000 UTC m=+1753.151676606 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-maas-api-tls") pod "maas-api-e2e-derive-8df77702-667bbf544d-h2hln" (UID: "3e2abd37-be9f-4cad-bb82-12e6f8f2768f") : secret "maas-api-e2e-derive-8df77702-serving-cert" not found Jul 08 18:47:11.900419 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.900412 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-openshift-service-ca\") pod \"maas-api-e2e-derive-8df77702-667bbf544d-h2hln\" (UID: \"3e2abd37-be9f-4cad-bb82-12e6f8f2768f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" Jul 08 18:47:11.901192 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.901172 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-openshift-service-ca\") pod \"maas-api-e2e-derive-8df77702-667bbf544d-h2hln\" (UID: \"3e2abd37-be9f-4cad-bb82-12e6f8f2768f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" Jul 08 18:47:11.908985 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:11.908960 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgnbw\" (UniqueName: \"kubernetes.io/projected/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-kube-api-access-dgnbw\") pod \"maas-api-e2e-derive-8df77702-667bbf544d-h2hln\" (UID: \"3e2abd37-be9f-4cad-bb82-12e6f8f2768f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" Jul 08 18:47:12.012668 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:12.012639 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:12.062310 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:12.062280 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:12.063283 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:12.063267 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:12.225871 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:12.225800 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:12.326388 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:12.326358 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:12.406246 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:12.406154 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-maas-api-tls\") pod \"maas-api-e2e-derive-8df77702-667bbf544d-h2hln\" (UID: \"3e2abd37-be9f-4cad-bb82-12e6f8f2768f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" Jul 08 18:47:12.408817 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:12.408792 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-maas-api-tls\") pod \"maas-api-e2e-derive-8df77702-667bbf544d-h2hln\" (UID: \"3e2abd37-be9f-4cad-bb82-12e6f8f2768f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" Jul 08 18:47:12.565268 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:12.565227 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" Jul 08 18:47:12.697885 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:12.697842 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln"] Jul 08 18:47:12.700492 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:47:12.700463 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e2abd37_be9f_4cad_bb82_12e6f8f2768f.slice/crio-06b64d206480a4604d49dc08ccfe480798650a2f5480d71b968239d6d0e82727 WatchSource:0}: Error finding container 06b64d206480a4604d49dc08ccfe480798650a2f5480d71b968239d6d0e82727: Status 404 returned error can't find the container with id 06b64d206480a4604d49dc08ccfe480798650a2f5480d71b968239d6d0e82727 Jul 08 18:47:13.067596 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:13.067570 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" event={"ID":"3e2abd37-be9f-4cad-bb82-12e6f8f2768f","Type":"ContainerStarted","Data":"06b64d206480a4604d49dc08ccfe480798650a2f5480d71b968239d6d0e82727"} Jul 08 18:47:14.073125 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:14.073085 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" event={"ID":"3e2abd37-be9f-4cad-bb82-12e6f8f2768f","Type":"ContainerStarted","Data":"466ce8edf86431f37c2a6225893a2fbe6e4bbbdeb20a85a1d880ea50809a5de3"} Jul 08 18:47:14.073632 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:14.073230 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" Jul 08 18:47:14.095802 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:14.095746 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" podStartSLOduration=2.739386131 podStartE2EDuration="3.095731047s" totalImagesPullingTime="356.344916ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:47:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:47:14.093584729 +0000 UTC m=+1754.844894205" watchObservedRunningTime="2026-07-08 18:47:14.095731047 +0000 UTC m=+1754.847040521" Jul 08 18:47:14.428381 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:14.428346 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:16.031233 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.031204 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:16.199342 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.199291 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf"] Jul 08 18:47:16.199590 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.199534 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" podUID="12521138-faa2-4657-99f5-a2c24efbd754" containerName="istio-proxy" containerID="cri-o://50344ceebc68f174820e431e13f2c5b221a90a1c4a0929fd5ba286f39d9c3a5a" gracePeriod=30 Jul 08 18:47:16.227160 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.227106 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:16.324146 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.324109 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:16.453340 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.453309 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln"] Jul 08 18:47:16.453505 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.453489 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" podUID="3e2abd37-be9f-4cad-bb82-12e6f8f2768f" containerName="maas-api" containerID="cri-o://466ce8edf86431f37c2a6225893a2fbe6e4bbbdeb20a85a1d880ea50809a5de3" gracePeriod=30 Jul 08 18:47:16.506753 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.506597 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:16.737992 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.737965 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" Jul 08 18:47:16.848733 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.848702 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-maas-api-tls\") pod \"3e2abd37-be9f-4cad-bb82-12e6f8f2768f\" (UID: \"3e2abd37-be9f-4cad-bb82-12e6f8f2768f\") " Jul 08 18:47:16.848983 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.848793 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgnbw\" (UniqueName: \"kubernetes.io/projected/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-kube-api-access-dgnbw\") pod \"3e2abd37-be9f-4cad-bb82-12e6f8f2768f\" (UID: \"3e2abd37-be9f-4cad-bb82-12e6f8f2768f\") " Jul 08 18:47:16.848983 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.848824 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-openshift-service-ca\") pod \"3e2abd37-be9f-4cad-bb82-12e6f8f2768f\" (UID: \"3e2abd37-be9f-4cad-bb82-12e6f8f2768f\") " Jul 08 18:47:16.849227 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.849198 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3e2abd37-be9f-4cad-bb82-12e6f8f2768f" (UID: "3e2abd37-be9f-4cad-bb82-12e6f8f2768f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:47:16.851035 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.851009 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3e2abd37-be9f-4cad-bb82-12e6f8f2768f" (UID: "3e2abd37-be9f-4cad-bb82-12e6f8f2768f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:47:16.851118 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.851011 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-kube-api-access-dgnbw" (OuterVolumeSpecName: "kube-api-access-dgnbw") pod "3e2abd37-be9f-4cad-bb82-12e6f8f2768f" (UID: "3e2abd37-be9f-4cad-bb82-12e6f8f2768f"). InnerVolumeSpecName "kube-api-access-dgnbw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:47:16.950399 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.950308 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-maas-api-tls\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:47:16.950399 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.950341 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgnbw\" (UniqueName: \"kubernetes.io/projected/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-kube-api-access-dgnbw\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:47:16.950399 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:16.950355 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2abd37-be9f-4cad-bb82-12e6f8f2768f-openshift-service-ca\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:47:17.092771 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:17.092709 2574 generic.go:356] "Generic (PLEG): container finished" podID="3e2abd37-be9f-4cad-bb82-12e6f8f2768f" containerID="466ce8edf86431f37c2a6225893a2fbe6e4bbbdeb20a85a1d880ea50809a5de3" exitCode=0 Jul 08 18:47:17.093293 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:17.092779 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" Jul 08 18:47:17.093293 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:17.092813 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" event={"ID":"3e2abd37-be9f-4cad-bb82-12e6f8f2768f","Type":"ContainerDied","Data":"466ce8edf86431f37c2a6225893a2fbe6e4bbbdeb20a85a1d880ea50809a5de3"} Jul 08 18:47:17.093293 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:17.092892 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln" event={"ID":"3e2abd37-be9f-4cad-bb82-12e6f8f2768f","Type":"ContainerDied","Data":"06b64d206480a4604d49dc08ccfe480798650a2f5480d71b968239d6d0e82727"} Jul 08 18:47:17.093293 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:17.092928 2574 scope.go:117] "RemoveContainer" containerID="466ce8edf86431f37c2a6225893a2fbe6e4bbbdeb20a85a1d880ea50809a5de3" Jul 08 18:47:17.117957 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:17.117933 2574 scope.go:117] "RemoveContainer" containerID="466ce8edf86431f37c2a6225893a2fbe6e4bbbdeb20a85a1d880ea50809a5de3" Jul 08 18:47:17.118324 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:47:17.118302 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"466ce8edf86431f37c2a6225893a2fbe6e4bbbdeb20a85a1d880ea50809a5de3\": container with ID starting with 466ce8edf86431f37c2a6225893a2fbe6e4bbbdeb20a85a1d880ea50809a5de3 not found: ID does not exist" containerID="466ce8edf86431f37c2a6225893a2fbe6e4bbbdeb20a85a1d880ea50809a5de3" Jul 08 18:47:17.118369 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:17.118334 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"466ce8edf86431f37c2a6225893a2fbe6e4bbbdeb20a85a1d880ea50809a5de3"} err="failed to get container status \"466ce8edf86431f37c2a6225893a2fbe6e4bbbdeb20a85a1d880ea50809a5de3\": rpc error: code = NotFound desc = could not find container \"466ce8edf86431f37c2a6225893a2fbe6e4bbbdeb20a85a1d880ea50809a5de3\": container with ID starting with 466ce8edf86431f37c2a6225893a2fbe6e4bbbdeb20a85a1d880ea50809a5de3 not found: ID does not exist" Jul 08 18:47:17.127874 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:17.127830 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln"] Jul 08 18:47:17.131310 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:17.131287 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8df77702-667bbf544d-h2hln"] Jul 08 18:47:17.770579 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:17.770547 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e2abd37-be9f-4cad-bb82-12e6f8f2768f" path="/var/lib/kubelet/pods/3e2abd37-be9f-4cad-bb82-12e6f8f2768f/volumes" Jul 08 18:47:18.128738 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:18.128706 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:22.071005 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.070980 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:22.123245 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.123217 2574 generic.go:356] "Generic (PLEG): container finished" podID="12521138-faa2-4657-99f5-a2c24efbd754" containerID="50344ceebc68f174820e431e13f2c5b221a90a1c4a0929fd5ba286f39d9c3a5a" exitCode=0 Jul 08 18:47:22.123397 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.123286 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" event={"ID":"12521138-faa2-4657-99f5-a2c24efbd754","Type":"ContainerDied","Data":"50344ceebc68f174820e431e13f2c5b221a90a1c4a0929fd5ba286f39d9c3a5a"} Jul 08 18:47:22.127813 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.127788 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:22.249483 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.249457 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:22.405672 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.405622 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12521138-faa2-4657-99f5-a2c24efbd754-istiod-ca-cert\") pod \"12521138-faa2-4657-99f5-a2c24efbd754\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " Jul 08 18:47:22.405904 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.405692 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12521138-faa2-4657-99f5-a2c24efbd754-istio-token\") pod \"12521138-faa2-4657-99f5-a2c24efbd754\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " Jul 08 18:47:22.405904 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.405745 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-workload-certs\") pod \"12521138-faa2-4657-99f5-a2c24efbd754\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " Jul 08 18:47:22.405904 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.405817 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12521138-faa2-4657-99f5-a2c24efbd754-istio-podinfo\") pod \"12521138-faa2-4657-99f5-a2c24efbd754\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " Jul 08 18:47:22.405904 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.405842 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-credential-socket\") pod \"12521138-faa2-4657-99f5-a2c24efbd754\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " Jul 08 18:47:22.405904 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.405897 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-workload-socket\") pod \"12521138-faa2-4657-99f5-a2c24efbd754\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " Jul 08 18:47:22.406169 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.405956 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hd4lb\" (UniqueName: \"kubernetes.io/projected/12521138-faa2-4657-99f5-a2c24efbd754-kube-api-access-hd4lb\") pod \"12521138-faa2-4657-99f5-a2c24efbd754\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " Jul 08 18:47:22.406169 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.405981 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-istio-envoy\") pod \"12521138-faa2-4657-99f5-a2c24efbd754\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " Jul 08 18:47:22.406169 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.406017 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-istio-data\") pod \"12521138-faa2-4657-99f5-a2c24efbd754\" (UID: \"12521138-faa2-4657-99f5-a2c24efbd754\") " Jul 08 18:47:22.406169 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.406094 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "12521138-faa2-4657-99f5-a2c24efbd754" (UID: "12521138-faa2-4657-99f5-a2c24efbd754"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:47:22.406169 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.406135 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12521138-faa2-4657-99f5-a2c24efbd754-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "12521138-faa2-4657-99f5-a2c24efbd754" (UID: "12521138-faa2-4657-99f5-a2c24efbd754"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:47:22.406169 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.406152 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "12521138-faa2-4657-99f5-a2c24efbd754" (UID: "12521138-faa2-4657-99f5-a2c24efbd754"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:47:22.406470 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.406303 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "12521138-faa2-4657-99f5-a2c24efbd754" (UID: "12521138-faa2-4657-99f5-a2c24efbd754"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:47:22.406470 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.406440 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12521138-faa2-4657-99f5-a2c24efbd754-istiod-ca-cert\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:47:22.406470 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.406457 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-workload-certs\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:47:22.406623 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.406470 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-credential-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:47:22.406623 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.406486 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-workload-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:47:22.407035 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.407012 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-istio-data" (OuterVolumeSpecName: "istio-data") pod "12521138-faa2-4657-99f5-a2c24efbd754" (UID: "12521138-faa2-4657-99f5-a2c24efbd754"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:47:22.408681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.408662 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/12521138-faa2-4657-99f5-a2c24efbd754-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "12521138-faa2-4657-99f5-a2c24efbd754" (UID: "12521138-faa2-4657-99f5-a2c24efbd754"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:47:22.408681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.408670 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12521138-faa2-4657-99f5-a2c24efbd754-istio-token" (OuterVolumeSpecName: "istio-token") pod "12521138-faa2-4657-99f5-a2c24efbd754" (UID: "12521138-faa2-4657-99f5-a2c24efbd754"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:47:22.408910 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.408889 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "12521138-faa2-4657-99f5-a2c24efbd754" (UID: "12521138-faa2-4657-99f5-a2c24efbd754"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:47:22.408988 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.408945 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12521138-faa2-4657-99f5-a2c24efbd754-kube-api-access-hd4lb" (OuterVolumeSpecName: "kube-api-access-hd4lb") pod "12521138-faa2-4657-99f5-a2c24efbd754" (UID: "12521138-faa2-4657-99f5-a2c24efbd754"). InnerVolumeSpecName "kube-api-access-hd4lb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:47:22.507660 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.507628 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12521138-faa2-4657-99f5-a2c24efbd754-istio-token\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:47:22.507660 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.507660 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12521138-faa2-4657-99f5-a2c24efbd754-istio-podinfo\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:47:22.507911 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.507675 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hd4lb\" (UniqueName: \"kubernetes.io/projected/12521138-faa2-4657-99f5-a2c24efbd754-kube-api-access-hd4lb\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:47:22.507911 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.507688 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-istio-envoy\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:47:22.507911 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.507696 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12521138-faa2-4657-99f5-a2c24efbd754-istio-data\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:47:22.583417 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.583388 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:22.683090 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.683007 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:22.785879 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:22.785829 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:23.129530 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:23.129500 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" event={"ID":"12521138-faa2-4657-99f5-a2c24efbd754","Type":"ContainerDied","Data":"a6e492f2d120da7e3b790e903e806f1dc5b5fd9ee4c15623e209fd2e5e395964"} Jul 08 18:47:23.129530 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:23.129521 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf" Jul 08 18:47:23.130032 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:23.129557 2574 scope.go:117] "RemoveContainer" containerID="50344ceebc68f174820e431e13f2c5b221a90a1c4a0929fd5ba286f39d9c3a5a" Jul 08 18:47:23.150816 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:23.150790 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf"] Jul 08 18:47:23.155145 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:23.155117 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-8df77702-openshift-default-86f58dcf5f-qhxwf"] Jul 08 18:47:23.770657 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:23.770616 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12521138-faa2-4657-99f5-a2c24efbd754" path="/var/lib/kubelet/pods/12521138-faa2-4657-99f5-a2c24efbd754/volumes" Jul 08 18:47:24.244681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:24.244650 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:30.025287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:30.025252 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:30.142399 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:30.142366 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:30.232196 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:30.232148 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:30.306873 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:30.306758 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:30.753558 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:30.753525 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:47:56.752630 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:47:56.752545 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:02.476219 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:02.476181 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:02.552150 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:02.552118 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:02.742373 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:02.742293 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:02.847681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:02.847649 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:18.618421 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:18.618383 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:24.311525 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:24.311480 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:24.416634 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:24.416597 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:24.511121 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:24.511078 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:35.049750 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:35.049718 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:35.872623 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:35.872586 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:41.568457 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:41.568423 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:41.628060 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:41.628021 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:41.717665 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:41.717619 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:41.813686 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:41.813653 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:41.911992 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:41.911897 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:42.679048 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:42.679017 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:42.779619 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:42.779585 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:42.878717 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:42.878687 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:44.548842 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:44.548807 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:50.234715 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:50.234675 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:50.305369 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:50.305339 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:50.935100 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:50.935061 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:56.621760 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:56.621725 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:56.722345 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:56.722314 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:56.876218 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:56.876119 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:57.255240 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.255148 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn"] Jul 08 18:48:57.255592 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.255574 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12521138-faa2-4657-99f5-a2c24efbd754" containerName="istio-proxy" Jul 08 18:48:57.255678 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.255595 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="12521138-faa2-4657-99f5-a2c24efbd754" containerName="istio-proxy" Jul 08 18:48:57.255678 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.255621 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e2abd37-be9f-4cad-bb82-12e6f8f2768f" containerName="maas-api" Jul 08 18:48:57.255678 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.255631 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e2abd37-be9f-4cad-bb82-12e6f8f2768f" containerName="maas-api" Jul 08 18:48:57.255984 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.255710 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="12521138-faa2-4657-99f5-a2c24efbd754" containerName="istio-proxy" Jul 08 18:48:57.255984 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.255721 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e2abd37-be9f-4cad-bb82-12e6f8f2768f" containerName="maas-api" Jul 08 18:48:57.258688 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.258668 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.263683 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.263658 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-b5ae16a2-openshift-default-dockercfg-fdclx\"" Jul 08 18:48:57.276861 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.276831 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:57.286518 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.286492 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn"] Jul 08 18:48:57.314939 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.314902 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-workload-certs\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.314939 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.314935 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-envoy\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.315259 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.314997 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-token\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.315259 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.315040 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-workload-socket\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.315259 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.315093 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb96beb9-076b-4652-8d00-5c53e593dfb7-istiod-ca-cert\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.315259 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.315144 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-data\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.315259 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.315169 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-credential-socket\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.315259 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.315193 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-podinfo\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.315259 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.315244 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w89rg\" (UniqueName: \"kubernetes.io/projected/fb96beb9-076b-4652-8d00-5c53e593dfb7-kube-api-access-w89rg\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.423287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.416843 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb96beb9-076b-4652-8d00-5c53e593dfb7-istiod-ca-cert\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.423287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.416951 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-data\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.423287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.416996 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-credential-socket\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.423287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.417030 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-podinfo\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.423287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.417061 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w89rg\" (UniqueName: \"kubernetes.io/projected/fb96beb9-076b-4652-8d00-5c53e593dfb7-kube-api-access-w89rg\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.423287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.417103 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-workload-certs\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.423287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.417135 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-envoy\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.423287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.417218 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-token\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.423287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.417268 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-workload-socket\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.423287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.417715 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-workload-socket\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.423287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.417760 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb96beb9-076b-4652-8d00-5c53e593dfb7-istiod-ca-cert\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.423287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.417988 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-data\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.423287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.418030 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-credential-socket\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.423287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.418252 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-workload-certs\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.423287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.421095 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-envoy\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.426135 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.426106 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w89rg\" (UniqueName: \"kubernetes.io/projected/fb96beb9-076b-4652-8d00-5c53e593dfb7-kube-api-access-w89rg\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.426135 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.426123 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-podinfo\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.426320 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.426142 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-token\") pod \"e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.569028 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.568999 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:57.629392 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.629364 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:48:57.711074 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.711048 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn"] Jul 08 18:48:57.713051 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:48:57.713019 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb96beb9_076b_4652_8d00_5c53e593dfb7.slice/crio-9db4691731a660d8f4799107f0185fe118d1afb835bb093ab3f09f8110f6100c WatchSource:0}: Error finding container 9db4691731a660d8f4799107f0185fe118d1afb835bb093ab3f09f8110f6100c: Status 404 returned error can't find the container with id 9db4691731a660d8f4799107f0185fe118d1afb835bb093ab3f09f8110f6100c Jul 08 18:48:57.715101 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.715058 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:48:57.715198 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.715149 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:48:57.715238 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:57.715193 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:48:58.569508 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:58.569475 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" event={"ID":"fb96beb9-076b-4652-8d00-5c53e593dfb7","Type":"ContainerStarted","Data":"ba9a54d273fdec5afc625bbb01f01c8baf515f73a10265641cb24bd236bb6e62"} Jul 08 18:48:58.569685 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:58.569528 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" event={"ID":"fb96beb9-076b-4652-8d00-5c53e593dfb7","Type":"ContainerStarted","Data":"9db4691731a660d8f4799107f0185fe118d1afb835bb093ab3f09f8110f6100c"} Jul 08 18:48:58.593179 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:58.593131 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" podStartSLOduration=1.593118179 podStartE2EDuration="1.593118179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:48:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:48:58.590169424 +0000 UTC m=+1859.341478898" watchObservedRunningTime="2026-07-08 18:48:58.593118179 +0000 UTC m=+1859.344427652" Jul 08 18:48:59.569688 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:59.569650 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:48:59.575186 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:48:59.575162 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:49:00.553423 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.553384 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:00.578909 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.578880 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:49:00.579789 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.579771 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:49:00.627927 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.627894 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp"] Jul 08 18:49:00.631979 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.631958 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" Jul 08 18:49:00.634236 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.634210 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-b5ae16a2-serving-cert\"" Jul 08 18:49:00.634370 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.634347 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rlgwv\"" Jul 08 18:49:00.643323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.643294 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp"] Jul 08 18:49:00.645819 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.645791 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:00.716207 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.716171 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:00.754517 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.754486 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/140af637-3da7-474f-b3b9-cae5e59c2ce3-maas-api-tls\") pod \"maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp\" (UID: \"140af637-3da7-474f-b3b9-cae5e59c2ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" Jul 08 18:49:00.754723 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.754536 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxhsl\" (UniqueName: \"kubernetes.io/projected/140af637-3da7-474f-b3b9-cae5e59c2ce3-kube-api-access-mxhsl\") pod \"maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp\" (UID: \"140af637-3da7-474f-b3b9-cae5e59c2ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" Jul 08 18:49:00.754723 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.754616 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/140af637-3da7-474f-b3b9-cae5e59c2ce3-openshift-service-ca\") pod \"maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp\" (UID: \"140af637-3da7-474f-b3b9-cae5e59c2ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" Jul 08 18:49:00.828195 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.828158 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:00.855840 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.855798 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/140af637-3da7-474f-b3b9-cae5e59c2ce3-maas-api-tls\") pod \"maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp\" (UID: \"140af637-3da7-474f-b3b9-cae5e59c2ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" Jul 08 18:49:00.856062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.855894 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxhsl\" (UniqueName: \"kubernetes.io/projected/140af637-3da7-474f-b3b9-cae5e59c2ce3-kube-api-access-mxhsl\") pod \"maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp\" (UID: \"140af637-3da7-474f-b3b9-cae5e59c2ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" Jul 08 18:49:00.856062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.855924 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/140af637-3da7-474f-b3b9-cae5e59c2ce3-openshift-service-ca\") pod \"maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp\" (UID: \"140af637-3da7-474f-b3b9-cae5e59c2ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" Jul 08 18:49:00.856591 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.856566 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/140af637-3da7-474f-b3b9-cae5e59c2ce3-openshift-service-ca\") pod \"maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp\" (UID: \"140af637-3da7-474f-b3b9-cae5e59c2ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" Jul 08 18:49:00.858489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.858465 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/140af637-3da7-474f-b3b9-cae5e59c2ce3-maas-api-tls\") pod \"maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp\" (UID: \"140af637-3da7-474f-b3b9-cae5e59c2ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" Jul 08 18:49:00.864042 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.864016 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxhsl\" (UniqueName: \"kubernetes.io/projected/140af637-3da7-474f-b3b9-cae5e59c2ce3-kube-api-access-mxhsl\") pod \"maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp\" (UID: \"140af637-3da7-474f-b3b9-cae5e59c2ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" Jul 08 18:49:00.945468 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:00.945438 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" Jul 08 18:49:01.084319 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:01.084291 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp"] Jul 08 18:49:01.086140 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:49:01.086107 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod140af637_3da7_474f_b3b9_cae5e59c2ce3.slice/crio-4c7ea6c76ae76fea8655c123f8f1e342464019764a14bd632929c41f1d603515 WatchSource:0}: Error finding container 4c7ea6c76ae76fea8655c123f8f1e342464019764a14bd632929c41f1d603515: Status 404 returned error can't find the container with id 4c7ea6c76ae76fea8655c123f8f1e342464019764a14bd632929c41f1d603515 Jul 08 18:49:01.218025 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:01.217992 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:01.317753 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:01.317722 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:01.584292 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:01.584195 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" event={"ID":"140af637-3da7-474f-b3b9-cae5e59c2ce3","Type":"ContainerStarted","Data":"4c7ea6c76ae76fea8655c123f8f1e342464019764a14bd632929c41f1d603515"} Jul 08 18:49:02.589889 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:02.589825 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" event={"ID":"140af637-3da7-474f-b3b9-cae5e59c2ce3","Type":"ContainerStarted","Data":"f13c8d88a6fbc58697a13c57414f2bf25b86ffee464b04c4e8cd30adcc38bab6"} Jul 08 18:49:02.590390 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:02.589910 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" Jul 08 18:49:02.609606 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:02.609557 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" podStartSLOduration=2.042413548 podStartE2EDuration="2.609541867s" totalImagesPullingTime="567.128319ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:49:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:49:02.607732185 +0000 UTC m=+1863.359041672" watchObservedRunningTime="2026-07-08 18:49:02.609541867 +0000 UTC m=+1863.360851340" Jul 08 18:49:05.082017 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:05.081984 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:05.755657 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:05.755620 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:05.874304 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:05.874265 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:06.221192 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:06.221153 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:08.608093 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:08.608055 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" Jul 08 18:49:11.993158 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:11.993126 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:12.052715 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:12.052686 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:12.406527 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:12.406491 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:12.712824 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:12.712746 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp"] Jul 08 18:49:12.713076 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:12.713045 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" podUID="140af637-3da7-474f-b3b9-cae5e59c2ce3" containerName="maas-api" containerID="cri-o://f13c8d88a6fbc58697a13c57414f2bf25b86ffee464b04c4e8cd30adcc38bab6" gracePeriod=30 Jul 08 18:49:12.737842 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:12.737815 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:12.836768 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:12.836739 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:12.953778 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:12.953754 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" Jul 08 18:49:13.083027 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.082989 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxhsl\" (UniqueName: \"kubernetes.io/projected/140af637-3da7-474f-b3b9-cae5e59c2ce3-kube-api-access-mxhsl\") pod \"140af637-3da7-474f-b3b9-cae5e59c2ce3\" (UID: \"140af637-3da7-474f-b3b9-cae5e59c2ce3\") " Jul 08 18:49:13.083465 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.083099 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/140af637-3da7-474f-b3b9-cae5e59c2ce3-openshift-service-ca\") pod \"140af637-3da7-474f-b3b9-cae5e59c2ce3\" (UID: \"140af637-3da7-474f-b3b9-cae5e59c2ce3\") " Jul 08 18:49:13.083465 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.083255 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/140af637-3da7-474f-b3b9-cae5e59c2ce3-maas-api-tls\") pod \"140af637-3da7-474f-b3b9-cae5e59c2ce3\" (UID: \"140af637-3da7-474f-b3b9-cae5e59c2ce3\") " Jul 08 18:49:13.083596 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.083457 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/140af637-3da7-474f-b3b9-cae5e59c2ce3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "140af637-3da7-474f-b3b9-cae5e59c2ce3" (UID: "140af637-3da7-474f-b3b9-cae5e59c2ce3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:49:13.085322 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.085299 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/140af637-3da7-474f-b3b9-cae5e59c2ce3-kube-api-access-mxhsl" (OuterVolumeSpecName: "kube-api-access-mxhsl") pod "140af637-3da7-474f-b3b9-cae5e59c2ce3" (UID: "140af637-3da7-474f-b3b9-cae5e59c2ce3"). InnerVolumeSpecName "kube-api-access-mxhsl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:49:13.085453 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.085429 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/140af637-3da7-474f-b3b9-cae5e59c2ce3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "140af637-3da7-474f-b3b9-cae5e59c2ce3" (UID: "140af637-3da7-474f-b3b9-cae5e59c2ce3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:49:13.184934 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.184896 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/140af637-3da7-474f-b3b9-cae5e59c2ce3-maas-api-tls\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:49:13.184934 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.184927 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxhsl\" (UniqueName: \"kubernetes.io/projected/140af637-3da7-474f-b3b9-cae5e59c2ce3-kube-api-access-mxhsl\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:49:13.184934 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.184942 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/140af637-3da7-474f-b3b9-cae5e59c2ce3-openshift-service-ca\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:49:13.644949 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.644914 2574 generic.go:356] "Generic (PLEG): container finished" podID="140af637-3da7-474f-b3b9-cae5e59c2ce3" containerID="f13c8d88a6fbc58697a13c57414f2bf25b86ffee464b04c4e8cd30adcc38bab6" exitCode=0 Jul 08 18:49:13.645155 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.644962 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" Jul 08 18:49:13.645155 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.645001 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" event={"ID":"140af637-3da7-474f-b3b9-cae5e59c2ce3","Type":"ContainerDied","Data":"f13c8d88a6fbc58697a13c57414f2bf25b86ffee464b04c4e8cd30adcc38bab6"} Jul 08 18:49:13.645155 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.645062 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp" event={"ID":"140af637-3da7-474f-b3b9-cae5e59c2ce3","Type":"ContainerDied","Data":"4c7ea6c76ae76fea8655c123f8f1e342464019764a14bd632929c41f1d603515"} Jul 08 18:49:13.645155 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.645094 2574 scope.go:117] "RemoveContainer" containerID="f13c8d88a6fbc58697a13c57414f2bf25b86ffee464b04c4e8cd30adcc38bab6" Jul 08 18:49:13.646677 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.646656 2574 generic.go:356] "Generic (PLEG): container finished" podID="fb96beb9-076b-4652-8d00-5c53e593dfb7" containerID="ba9a54d273fdec5afc625bbb01f01c8baf515f73a10265641cb24bd236bb6e62" exitCode=0 Jul 08 18:49:13.646782 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.646720 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" event={"ID":"fb96beb9-076b-4652-8d00-5c53e593dfb7","Type":"ContainerDied","Data":"ba9a54d273fdec5afc625bbb01f01c8baf515f73a10265641cb24bd236bb6e62"} Jul 08 18:49:13.647190 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.647173 2574 scope.go:117] "RemoveContainer" containerID="ba9a54d273fdec5afc625bbb01f01c8baf515f73a10265641cb24bd236bb6e62" Jul 08 18:49:13.648298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.648266 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:49:13.648434 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.648352 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:49:13.648434 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.648395 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:49:13.658281 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.658260 2574 scope.go:117] "RemoveContainer" containerID="f13c8d88a6fbc58697a13c57414f2bf25b86ffee464b04c4e8cd30adcc38bab6" Jul 08 18:49:13.658561 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:49:13.658540 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f13c8d88a6fbc58697a13c57414f2bf25b86ffee464b04c4e8cd30adcc38bab6\": container with ID starting with f13c8d88a6fbc58697a13c57414f2bf25b86ffee464b04c4e8cd30adcc38bab6 not found: ID does not exist" containerID="f13c8d88a6fbc58697a13c57414f2bf25b86ffee464b04c4e8cd30adcc38bab6" Jul 08 18:49:13.658643 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.658571 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f13c8d88a6fbc58697a13c57414f2bf25b86ffee464b04c4e8cd30adcc38bab6"} err="failed to get container status \"f13c8d88a6fbc58697a13c57414f2bf25b86ffee464b04c4e8cd30adcc38bab6\": rpc error: code = NotFound desc = could not find container \"f13c8d88a6fbc58697a13c57414f2bf25b86ffee464b04c4e8cd30adcc38bab6\": container with ID starting with f13c8d88a6fbc58697a13c57414f2bf25b86ffee464b04c4e8cd30adcc38bab6 not found: ID does not exist" Jul 08 18:49:13.686934 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.686902 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp"] Jul 08 18:49:13.690495 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.690467 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b5ae16a2-6657bc48f8-bvpdp"] Jul 08 18:49:13.772820 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:13.772785 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="140af637-3da7-474f-b3b9-cae5e59c2ce3" path="/var/lib/kubelet/pods/140af637-3da7-474f-b3b9-cae5e59c2ce3/volumes" Jul 08 18:49:14.655482 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:14.655453 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" event={"ID":"fb96beb9-076b-4652-8d00-5c53e593dfb7","Type":"ContainerStarted","Data":"b7b27b9ab4a497e0da6be10460ed08b52c4dc7ef009602f22685e967113878da"} Jul 08 18:49:15.569348 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:15.569306 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:49:15.570496 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:15.570468 2574 patch_prober.go:28] interesting pod/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 08 18:49:15.570619 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:15.570521 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" podUID="fb96beb9-076b-4652-8d00-5c53e593dfb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 08 18:49:15.579672 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:15.579653 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:49:16.569789 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:16.569752 2574 patch_prober.go:28] interesting pod/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 08 18:49:16.570181 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:16.569813 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" podUID="fb96beb9-076b-4652-8d00-5c53e593dfb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 08 18:49:17.570137 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:17.570103 2574 patch_prober.go:28] interesting pod/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 08 18:49:17.570539 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:17.570160 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" podUID="fb96beb9-076b-4652-8d00-5c53e593dfb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 08 18:49:18.573391 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:18.573363 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:49:18.574408 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:18.574383 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:49:18.574570 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:18.574477 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:18.683748 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:18.683714 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:18.848611 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:18.848534 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn"] Jul 08 18:49:18.870342 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:18.870309 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:19.233963 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:19.233874 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:19.329056 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:19.329025 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:19.463335 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:19.463290 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:19.680486 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:19.680442 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" podUID="fb96beb9-076b-4652-8d00-5c53e593dfb7" containerName="istio-proxy" containerID="cri-o://b7b27b9ab4a497e0da6be10460ed08b52c4dc7ef009602f22685e967113878da" gracePeriod=30 Jul 08 18:49:24.940948 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:24.940925 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:49:25.005487 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.005402 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb96beb9-076b-4652-8d00-5c53e593dfb7-istiod-ca-cert\") pod \"fb96beb9-076b-4652-8d00-5c53e593dfb7\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " Jul 08 18:49:25.005487 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.005442 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-workload-certs\") pod \"fb96beb9-076b-4652-8d00-5c53e593dfb7\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " Jul 08 18:49:25.005487 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.005471 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-credential-socket\") pod \"fb96beb9-076b-4652-8d00-5c53e593dfb7\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " Jul 08 18:49:25.005739 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.005497 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w89rg\" (UniqueName: \"kubernetes.io/projected/fb96beb9-076b-4652-8d00-5c53e593dfb7-kube-api-access-w89rg\") pod \"fb96beb9-076b-4652-8d00-5c53e593dfb7\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " Jul 08 18:49:25.005739 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.005526 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-workload-socket\") pod \"fb96beb9-076b-4652-8d00-5c53e593dfb7\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " Jul 08 18:49:25.005739 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.005564 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-podinfo\") pod \"fb96beb9-076b-4652-8d00-5c53e593dfb7\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " Jul 08 18:49:25.005739 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.005617 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-envoy\") pod \"fb96beb9-076b-4652-8d00-5c53e593dfb7\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " Jul 08 18:49:25.005739 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.005663 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-token\") pod \"fb96beb9-076b-4652-8d00-5c53e593dfb7\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " Jul 08 18:49:25.005739 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.005696 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-data\") pod \"fb96beb9-076b-4652-8d00-5c53e593dfb7\" (UID: \"fb96beb9-076b-4652-8d00-5c53e593dfb7\") " Jul 08 18:49:25.006073 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.005767 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fb96beb9-076b-4652-8d00-5c53e593dfb7" (UID: "fb96beb9-076b-4652-8d00-5c53e593dfb7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:49:25.006073 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.005786 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fb96beb9-076b-4652-8d00-5c53e593dfb7" (UID: "fb96beb9-076b-4652-8d00-5c53e593dfb7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:49:25.006073 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.005954 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fb96beb9-076b-4652-8d00-5c53e593dfb7" (UID: "fb96beb9-076b-4652-8d00-5c53e593dfb7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:49:25.006215 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.006117 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-workload-certs\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:49:25.006215 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.006137 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-credential-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:49:25.006215 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.006144 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb96beb9-076b-4652-8d00-5c53e593dfb7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fb96beb9-076b-4652-8d00-5c53e593dfb7" (UID: "fb96beb9-076b-4652-8d00-5c53e593dfb7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:49:25.006215 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.006155 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-workload-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:49:25.006976 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.006953 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-data" (OuterVolumeSpecName: "istio-data") pod "fb96beb9-076b-4652-8d00-5c53e593dfb7" (UID: "fb96beb9-076b-4652-8d00-5c53e593dfb7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:49:25.007918 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.007895 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fb96beb9-076b-4652-8d00-5c53e593dfb7" (UID: "fb96beb9-076b-4652-8d00-5c53e593dfb7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:49:25.008093 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.008064 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fb96beb9-076b-4652-8d00-5c53e593dfb7" (UID: "fb96beb9-076b-4652-8d00-5c53e593dfb7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:49:25.008466 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.008445 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb96beb9-076b-4652-8d00-5c53e593dfb7-kube-api-access-w89rg" (OuterVolumeSpecName: "kube-api-access-w89rg") pod "fb96beb9-076b-4652-8d00-5c53e593dfb7" (UID: "fb96beb9-076b-4652-8d00-5c53e593dfb7"). InnerVolumeSpecName "kube-api-access-w89rg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:49:25.008539 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.008469 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-token" (OuterVolumeSpecName: "istio-token") pod "fb96beb9-076b-4652-8d00-5c53e593dfb7" (UID: "fb96beb9-076b-4652-8d00-5c53e593dfb7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:49:25.107330 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.107293 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb96beb9-076b-4652-8d00-5c53e593dfb7-istiod-ca-cert\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:49:25.107330 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.107323 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w89rg\" (UniqueName: \"kubernetes.io/projected/fb96beb9-076b-4652-8d00-5c53e593dfb7-kube-api-access-w89rg\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:49:25.107330 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.107334 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-podinfo\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:49:25.107572 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.107343 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-envoy\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:49:25.107572 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.107351 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-token\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:49:25.107572 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.107358 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb96beb9-076b-4652-8d00-5c53e593dfb7-istio-data\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:49:25.115538 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.115501 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:25.172254 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.172197 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:25.252493 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.252450 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:25.349454 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.349403 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:25.468838 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.468807 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:25.710363 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.710278 2574 generic.go:356] "Generic (PLEG): container finished" podID="fb96beb9-076b-4652-8d00-5c53e593dfb7" containerID="b7b27b9ab4a497e0da6be10460ed08b52c4dc7ef009602f22685e967113878da" exitCode=0 Jul 08 18:49:25.710363 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.710327 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" Jul 08 18:49:25.710542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.710366 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" event={"ID":"fb96beb9-076b-4652-8d00-5c53e593dfb7","Type":"ContainerDied","Data":"b7b27b9ab4a497e0da6be10460ed08b52c4dc7ef009602f22685e967113878da"} Jul 08 18:49:25.710542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.710419 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn" event={"ID":"fb96beb9-076b-4652-8d00-5c53e593dfb7","Type":"ContainerDied","Data":"9db4691731a660d8f4799107f0185fe118d1afb835bb093ab3f09f8110f6100c"} Jul 08 18:49:25.710542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.710444 2574 scope.go:117] "RemoveContainer" containerID="b7b27b9ab4a497e0da6be10460ed08b52c4dc7ef009602f22685e967113878da" Jul 08 18:49:25.719916 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.719900 2574 scope.go:117] "RemoveContainer" containerID="ba9a54d273fdec5afc625bbb01f01c8baf515f73a10265641cb24bd236bb6e62" Jul 08 18:49:25.727682 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.727667 2574 scope.go:117] "RemoveContainer" containerID="b7b27b9ab4a497e0da6be10460ed08b52c4dc7ef009602f22685e967113878da" Jul 08 18:49:25.727961 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:49:25.727942 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7b27b9ab4a497e0da6be10460ed08b52c4dc7ef009602f22685e967113878da\": container with ID starting with b7b27b9ab4a497e0da6be10460ed08b52c4dc7ef009602f22685e967113878da not found: ID does not exist" containerID="b7b27b9ab4a497e0da6be10460ed08b52c4dc7ef009602f22685e967113878da" Jul 08 18:49:25.728003 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.727971 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7b27b9ab4a497e0da6be10460ed08b52c4dc7ef009602f22685e967113878da"} err="failed to get container status \"b7b27b9ab4a497e0da6be10460ed08b52c4dc7ef009602f22685e967113878da\": rpc error: code = NotFound desc = could not find container \"b7b27b9ab4a497e0da6be10460ed08b52c4dc7ef009602f22685e967113878da\": container with ID starting with b7b27b9ab4a497e0da6be10460ed08b52c4dc7ef009602f22685e967113878da not found: ID does not exist" Jul 08 18:49:25.728003 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.727989 2574 scope.go:117] "RemoveContainer" containerID="ba9a54d273fdec5afc625bbb01f01c8baf515f73a10265641cb24bd236bb6e62" Jul 08 18:49:25.728184 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:49:25.728167 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba9a54d273fdec5afc625bbb01f01c8baf515f73a10265641cb24bd236bb6e62\": container with ID starting with ba9a54d273fdec5afc625bbb01f01c8baf515f73a10265641cb24bd236bb6e62 not found: ID does not exist" containerID="ba9a54d273fdec5afc625bbb01f01c8baf515f73a10265641cb24bd236bb6e62" Jul 08 18:49:25.728221 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.728191 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba9a54d273fdec5afc625bbb01f01c8baf515f73a10265641cb24bd236bb6e62"} err="failed to get container status \"ba9a54d273fdec5afc625bbb01f01c8baf515f73a10265641cb24bd236bb6e62\": rpc error: code = NotFound desc = could not find container \"ba9a54d273fdec5afc625bbb01f01c8baf515f73a10265641cb24bd236bb6e62\": container with ID starting with ba9a54d273fdec5afc625bbb01f01c8baf515f73a10265641cb24bd236bb6e62 not found: ID does not exist" Jul 08 18:49:25.732328 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.732306 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn"] Jul 08 18:49:25.736109 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.736089 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-b5ae16a2-openshift-default-c567fdb47-2tqmn"] Jul 08 18:49:25.770652 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:25.770624 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb96beb9-076b-4652-8d00-5c53e593dfb7" path="/var/lib/kubelet/pods/fb96beb9-076b-4652-8d00-5c53e593dfb7/volumes" Jul 08 18:49:31.095799 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:31.095757 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:31.197087 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:31.197041 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:37.979189 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:37.979154 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:38.285065 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.284979 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj"] Jul 08 18:49:38.285399 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.285365 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb96beb9-076b-4652-8d00-5c53e593dfb7" containerName="istio-proxy" Jul 08 18:49:38.285399 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.285381 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb96beb9-076b-4652-8d00-5c53e593dfb7" containerName="istio-proxy" Jul 08 18:49:38.285399 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.285398 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="140af637-3da7-474f-b3b9-cae5e59c2ce3" containerName="maas-api" Jul 08 18:49:38.285666 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.285404 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="140af637-3da7-474f-b3b9-cae5e59c2ce3" containerName="maas-api" Jul 08 18:49:38.285666 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.285464 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="140af637-3da7-474f-b3b9-cae5e59c2ce3" containerName="maas-api" Jul 08 18:49:38.285666 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.285472 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb96beb9-076b-4652-8d00-5c53e593dfb7" containerName="istio-proxy" Jul 08 18:49:38.285666 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.285481 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb96beb9-076b-4652-8d00-5c53e593dfb7" containerName="istio-proxy" Jul 08 18:49:38.288618 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.288600 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.291135 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.291101 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f330a831-openshift-default-dockercfg-8tx86\"" Jul 08 18:49:38.301955 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.301932 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj"] Jul 08 18:49:38.317126 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.317091 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:38.437086 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.437050 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-credential-socket\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.437288 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.437106 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-podinfo\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.437288 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.437127 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-envoy\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.437288 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.437173 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istiod-ca-cert\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.437288 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.437200 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-workload-socket\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.437288 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.437217 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-data\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.437288 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.437237 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fmhz\" (UniqueName: \"kubernetes.io/projected/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-kube-api-access-6fmhz\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.437546 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.437286 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-workload-certs\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.437546 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.437353 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-token\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.513974 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.513940 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:38.538730 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.538625 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istiod-ca-cert\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.538730 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.538698 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-workload-socket\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.538998 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.538730 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-data\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.538998 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.538841 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fmhz\" (UniqueName: \"kubernetes.io/projected/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-kube-api-access-6fmhz\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.538998 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.538896 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-workload-certs\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.538998 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.538946 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-token\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.539222 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.539104 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-credential-socket\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.539222 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.539179 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-podinfo\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.539222 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.539192 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-workload-socket\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.539222 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.539208 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-envoy\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.539429 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.539264 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-data\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.539484 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.539425 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-workload-certs\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.539545 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.539503 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istiod-ca-cert\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.539600 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.539545 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-credential-socket\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.541625 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.541602 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-envoy\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.541861 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.541825 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-podinfo\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.547386 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.547362 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-token\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.547642 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.547617 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fmhz\" (UniqueName: \"kubernetes.io/projected/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-kube-api-access-6fmhz\") pod \"e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.599320 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.599284 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:38.753408 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.753379 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj"] Jul 08 18:49:38.754902 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:49:38.754878 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab7e8831_cd3e_49dc_8c8c_a838d4dc6414.slice/crio-a0035f91d98193ab4114f5ffe2af509f2c4fa72a7043644461ab5c2984b2eddd WatchSource:0}: Error finding container a0035f91d98193ab4114f5ffe2af509f2c4fa72a7043644461ab5c2984b2eddd: Status 404 returned error can't find the container with id a0035f91d98193ab4114f5ffe2af509f2c4fa72a7043644461ab5c2984b2eddd Jul 08 18:49:38.757146 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.757112 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:49:38.757210 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.757179 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:49:38.757258 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.757209 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:49:38.776735 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:38.776701 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" event={"ID":"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414","Type":"ContainerStarted","Data":"a0035f91d98193ab4114f5ffe2af509f2c4fa72a7043644461ab5c2984b2eddd"} Jul 08 18:49:39.782922 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:39.782886 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" event={"ID":"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414","Type":"ContainerStarted","Data":"24f0f6fa470d783a3d4ed450f3e4e672508d70e275d091c6392c2106963244ee"} Jul 08 18:49:39.802576 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:39.802492 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" podStartSLOduration=1.8024731200000002 podStartE2EDuration="1.80247312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:49:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:49:39.800754961 +0000 UTC m=+1900.552064425" watchObservedRunningTime="2026-07-08 18:49:39.80247312 +0000 UTC m=+1900.553782594" Jul 08 18:49:40.599583 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:40.599542 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:40.604897 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:40.604869 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:40.787866 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:40.787820 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:40.788814 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:40.788796 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:41.724630 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:41.724589 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:41.803190 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:41.803156 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722"] Jul 08 18:49:41.803793 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:41.803771 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb96beb9-076b-4652-8d00-5c53e593dfb7" containerName="istio-proxy" Jul 08 18:49:41.803882 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:41.803799 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb96beb9-076b-4652-8d00-5c53e593dfb7" containerName="istio-proxy" Jul 08 18:49:41.807343 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:41.807324 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" Jul 08 18:49:41.810071 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:41.810050 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rlgwv\"" Jul 08 18:49:41.810071 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:41.810072 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f330a831-serving-cert\"" Jul 08 18:49:41.815838 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:41.815799 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722"] Jul 08 18:49:41.820982 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:41.820959 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:41.886017 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:41.885987 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:41.977472 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:41.977383 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qbv4\" (UniqueName: \"kubernetes.io/projected/b2d302da-c5eb-48af-9079-191af30f4d7e-kube-api-access-7qbv4\") pod \"maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722\" (UID: \"b2d302da-c5eb-48af-9079-191af30f4d7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" Jul 08 18:49:41.977472 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:41.977426 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2d302da-c5eb-48af-9079-191af30f4d7e-maas-api-tls\") pod \"maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722\" (UID: \"b2d302da-c5eb-48af-9079-191af30f4d7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" Jul 08 18:49:41.977661 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:41.977506 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2d302da-c5eb-48af-9079-191af30f4d7e-openshift-service-ca\") pod \"maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722\" (UID: \"b2d302da-c5eb-48af-9079-191af30f4d7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" Jul 08 18:49:42.041704 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:42.041673 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:42.078098 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:42.078062 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2d302da-c5eb-48af-9079-191af30f4d7e-openshift-service-ca\") pod \"maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722\" (UID: \"b2d302da-c5eb-48af-9079-191af30f4d7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" Jul 08 18:49:42.078294 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:42.078171 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qbv4\" (UniqueName: \"kubernetes.io/projected/b2d302da-c5eb-48af-9079-191af30f4d7e-kube-api-access-7qbv4\") pod \"maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722\" (UID: \"b2d302da-c5eb-48af-9079-191af30f4d7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" Jul 08 18:49:42.078294 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:42.078191 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2d302da-c5eb-48af-9079-191af30f4d7e-maas-api-tls\") pod \"maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722\" (UID: \"b2d302da-c5eb-48af-9079-191af30f4d7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" Jul 08 18:49:42.078779 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:42.078738 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2d302da-c5eb-48af-9079-191af30f4d7e-openshift-service-ca\") pod \"maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722\" (UID: \"b2d302da-c5eb-48af-9079-191af30f4d7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" Jul 08 18:49:42.080721 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:42.080698 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2d302da-c5eb-48af-9079-191af30f4d7e-maas-api-tls\") pod \"maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722\" (UID: \"b2d302da-c5eb-48af-9079-191af30f4d7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" Jul 08 18:49:42.089771 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:42.089748 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qbv4\" (UniqueName: \"kubernetes.io/projected/b2d302da-c5eb-48af-9079-191af30f4d7e-kube-api-access-7qbv4\") pod \"maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722\" (UID: \"b2d302da-c5eb-48af-9079-191af30f4d7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" Jul 08 18:49:42.121809 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:42.121766 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" Jul 08 18:49:42.260062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:42.260036 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722"] Jul 08 18:49:42.262168 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:49:42.262142 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d302da_c5eb_48af_9079_191af30f4d7e.slice/crio-c11cf791a5e9667a075fa05fa60f5dd4b258b65c6603172aa21e94aa95617b14 WatchSource:0}: Error finding container c11cf791a5e9667a075fa05fa60f5dd4b258b65c6603172aa21e94aa95617b14: Status 404 returned error can't find the container with id c11cf791a5e9667a075fa05fa60f5dd4b258b65c6603172aa21e94aa95617b14 Jul 08 18:49:42.393394 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:42.393358 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:42.493429 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:42.493399 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:42.799483 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:42.799447 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" event={"ID":"b2d302da-c5eb-48af-9079-191af30f4d7e","Type":"ContainerStarted","Data":"bbb1383a075102b0bdc37e59597a17bbf395207ce82ccdb54f271a746dae37c8"} Jul 08 18:49:42.799483 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:42.799501 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" event={"ID":"b2d302da-c5eb-48af-9079-191af30f4d7e","Type":"ContainerStarted","Data":"c11cf791a5e9667a075fa05fa60f5dd4b258b65c6603172aa21e94aa95617b14"} Jul 08 18:49:42.799755 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:42.799543 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" Jul 08 18:49:42.816650 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:42.816578 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" podStartSLOduration=1.489282948 podStartE2EDuration="1.816562352s" totalImagesPullingTime="327.279404ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:49:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:49:42.815478252 +0000 UTC m=+1903.566787744" watchObservedRunningTime="2026-07-08 18:49:42.816562352 +0000 UTC m=+1903.567871864" Jul 08 18:49:46.260188 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:46.260154 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:47.387748 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:47.387718 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:47.485233 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:47.485201 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:48.810101 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:48.810067 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" Jul 08 18:49:53.513484 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:53.513408 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:56.487182 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:56.486539 2574 patch_prober.go:28] interesting pod/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": read tcp 10.133.0.2:46732->10.133.0.34:15021: read: connection reset by peer" start-of-body= Jul 08 18:49:56.487182 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:56.486637 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" podUID="ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": read tcp 10.133.0.2:46732->10.133.0.34:15021: read: connection reset by peer" Jul 08 18:49:56.877176 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:56.877137 2574 generic.go:356] "Generic (PLEG): container finished" podID="ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" containerID="24f0f6fa470d783a3d4ed450f3e4e672508d70e275d091c6392c2106963244ee" exitCode=0 Jul 08 18:49:56.877361 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:56.877216 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" event={"ID":"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414","Type":"ContainerDied","Data":"24f0f6fa470d783a3d4ed450f3e4e672508d70e275d091c6392c2106963244ee"} Jul 08 18:49:56.877656 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:56.877640 2574 scope.go:117] "RemoveContainer" containerID="24f0f6fa470d783a3d4ed450f3e4e672508d70e275d091c6392c2106963244ee" Jul 08 18:49:56.878686 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:56.878649 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:49:56.878776 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:56.878719 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:49:56.878776 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:56.878751 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:49:57.883178 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:57.883135 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" event={"ID":"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414","Type":"ContainerStarted","Data":"cb5ba09b1a6356a0a592f32b241e88a61d9b38b26dafc65f180fa71e57ffd3a0"} Jul 08 18:49:58.206398 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:58.206285 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:49:58.600517 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:58.600482 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:49:58.601179 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:58.600811 2574 patch_prober.go:28] interesting pod/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 08 18:49:58.601179 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:58.600907 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" podUID="ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 08 18:49:59.600164 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:59.600115 2574 patch_prober.go:28] interesting pod/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 08 18:49:59.600664 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:49:59.600201 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" podUID="ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 08 18:50:00.600041 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:00.599998 2574 patch_prober.go:28] interesting pod/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 08 18:50:00.600332 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:00.600302 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" podUID="ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 08 18:50:01.600299 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:01.600249 2574 patch_prober.go:28] interesting pod/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 08 18:50:01.600562 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:01.600324 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" podUID="ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 08 18:50:02.602997 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:02.602966 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:50:02.603387 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:02.603233 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:50:02.603814 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:02.603800 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:50:03.106864 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:03.106825 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:50:03.208211 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:03.208182 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:50:23.954479 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:23.954443 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:50:24.273684 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:24.273587 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722"] Jul 08 18:50:24.273920 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:24.273870 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" podUID="b2d302da-c5eb-48af-9079-191af30f4d7e" containerName="maas-api" containerID="cri-o://bbb1383a075102b0bdc37e59597a17bbf395207ce82ccdb54f271a746dae37c8" gracePeriod=30 Jul 08 18:50:24.299951 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:24.299907 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:50:24.529480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:24.529419 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" Jul 08 18:50:24.631873 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:24.631820 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2d302da-c5eb-48af-9079-191af30f4d7e-maas-api-tls\") pod \"b2d302da-c5eb-48af-9079-191af30f4d7e\" (UID: \"b2d302da-c5eb-48af-9079-191af30f4d7e\") " Jul 08 18:50:24.632061 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:24.631909 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qbv4\" (UniqueName: \"kubernetes.io/projected/b2d302da-c5eb-48af-9079-191af30f4d7e-kube-api-access-7qbv4\") pod \"b2d302da-c5eb-48af-9079-191af30f4d7e\" (UID: \"b2d302da-c5eb-48af-9079-191af30f4d7e\") " Jul 08 18:50:24.632061 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:24.631961 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2d302da-c5eb-48af-9079-191af30f4d7e-openshift-service-ca\") pod \"b2d302da-c5eb-48af-9079-191af30f4d7e\" (UID: \"b2d302da-c5eb-48af-9079-191af30f4d7e\") " Jul 08 18:50:24.632338 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:24.632314 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2d302da-c5eb-48af-9079-191af30f4d7e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b2d302da-c5eb-48af-9079-191af30f4d7e" (UID: "b2d302da-c5eb-48af-9079-191af30f4d7e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:50:24.633970 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:24.633946 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2d302da-c5eb-48af-9079-191af30f4d7e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b2d302da-c5eb-48af-9079-191af30f4d7e" (UID: "b2d302da-c5eb-48af-9079-191af30f4d7e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:50:24.634072 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:24.634035 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d302da-c5eb-48af-9079-191af30f4d7e-kube-api-access-7qbv4" (OuterVolumeSpecName: "kube-api-access-7qbv4") pod "b2d302da-c5eb-48af-9079-191af30f4d7e" (UID: "b2d302da-c5eb-48af-9079-191af30f4d7e"). InnerVolumeSpecName "kube-api-access-7qbv4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:50:24.732751 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:24.732687 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2d302da-c5eb-48af-9079-191af30f4d7e-maas-api-tls\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:50:24.732751 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:24.732742 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qbv4\" (UniqueName: \"kubernetes.io/projected/b2d302da-c5eb-48af-9079-191af30f4d7e-kube-api-access-7qbv4\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:50:24.732751 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:24.732754 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2d302da-c5eb-48af-9079-191af30f4d7e-openshift-service-ca\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:50:25.032098 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:25.032060 2574 generic.go:356] "Generic (PLEG): container finished" podID="b2d302da-c5eb-48af-9079-191af30f4d7e" containerID="bbb1383a075102b0bdc37e59597a17bbf395207ce82ccdb54f271a746dae37c8" exitCode=0 Jul 08 18:50:25.032532 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:25.032103 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" Jul 08 18:50:25.032532 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:25.032147 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" event={"ID":"b2d302da-c5eb-48af-9079-191af30f4d7e","Type":"ContainerDied","Data":"bbb1383a075102b0bdc37e59597a17bbf395207ce82ccdb54f271a746dae37c8"} Jul 08 18:50:25.032532 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:25.032198 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722" event={"ID":"b2d302da-c5eb-48af-9079-191af30f4d7e","Type":"ContainerDied","Data":"c11cf791a5e9667a075fa05fa60f5dd4b258b65c6603172aa21e94aa95617b14"} Jul 08 18:50:25.032532 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:25.032222 2574 scope.go:117] "RemoveContainer" containerID="bbb1383a075102b0bdc37e59597a17bbf395207ce82ccdb54f271a746dae37c8" Jul 08 18:50:25.041498 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:25.041482 2574 scope.go:117] "RemoveContainer" containerID="bbb1383a075102b0bdc37e59597a17bbf395207ce82ccdb54f271a746dae37c8" Jul 08 18:50:25.041727 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:50:25.041712 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbb1383a075102b0bdc37e59597a17bbf395207ce82ccdb54f271a746dae37c8\": container with ID starting with bbb1383a075102b0bdc37e59597a17bbf395207ce82ccdb54f271a746dae37c8 not found: ID does not exist" containerID="bbb1383a075102b0bdc37e59597a17bbf395207ce82ccdb54f271a746dae37c8" Jul 08 18:50:25.041769 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:25.041736 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbb1383a075102b0bdc37e59597a17bbf395207ce82ccdb54f271a746dae37c8"} err="failed to get container status \"bbb1383a075102b0bdc37e59597a17bbf395207ce82ccdb54f271a746dae37c8\": rpc error: code = NotFound desc = could not find container \"bbb1383a075102b0bdc37e59597a17bbf395207ce82ccdb54f271a746dae37c8\": container with ID starting with bbb1383a075102b0bdc37e59597a17bbf395207ce82ccdb54f271a746dae37c8 not found: ID does not exist" Jul 08 18:50:25.053026 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:25.053000 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722"] Jul 08 18:50:25.057581 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:25.057555 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f330a831-5fbc9c99bd-bq722"] Jul 08 18:50:25.770683 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:25.770648 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2d302da-c5eb-48af-9079-191af30f4d7e" path="/var/lib/kubelet/pods/b2d302da-c5eb-48af-9079-191af30f4d7e/volumes" Jul 08 18:50:30.042374 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:30.042336 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:50:35.072396 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:50:35.072356 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:11.249152 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:11.249116 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:11.990061 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:11.990027 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:17.648394 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:17.648358 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:23.991663 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:23.991626 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:31.740700 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:31.740660 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:31.840916 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:31.840874 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:32.062444 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:32.062406 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj"] Jul 08 18:51:32.062669 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:32.062646 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" podUID="ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" containerName="istio-proxy" containerID="cri-o://cb5ba09b1a6356a0a592f32b241e88a61d9b38b26dafc65f180fa71e57ffd3a0" gracePeriod=30 Jul 08 18:51:32.098314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:32.098282 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:32.605135 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:32.605100 2574 patch_prober.go:28] interesting pod/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 18:51:32.605298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:32.605175 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" podUID="ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 18:51:33.384886 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:33.384817 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:33.480968 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:33.480931 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:33.581211 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:33.581176 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:34.377171 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:34.377136 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:34.475116 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:34.475082 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:34.579966 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:34.579932 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:36.031682 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:36.031646 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:37.310746 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.310720 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:51:37.346480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.346406 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-credential-socket\") pod \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " Jul 08 18:51:37.346480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.346443 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-workload-socket\") pod \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " Jul 08 18:51:37.346708 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.346507 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istiod-ca-cert\") pod \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " Jul 08 18:51:37.346708 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.346528 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-data\") pod \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " Jul 08 18:51:37.346708 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.346547 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-token\") pod \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " Jul 08 18:51:37.346708 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.346580 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-workload-certs\") pod \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " Jul 08 18:51:37.346708 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.346634 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-podinfo\") pod \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " Jul 08 18:51:37.346708 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.346667 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fmhz\" (UniqueName: \"kubernetes.io/projected/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-kube-api-access-6fmhz\") pod \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " Jul 08 18:51:37.346708 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.346698 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-envoy\") pod \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\" (UID: \"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414\") " Jul 08 18:51:37.347098 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.346713 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" (UID: "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:51:37.347098 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.346722 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" (UID: "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:51:37.347098 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.346941 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" (UID: "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:51:37.347098 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.347031 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-workload-certs\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:51:37.347098 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.347044 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-credential-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:51:37.347098 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.347054 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-workload-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:51:37.347652 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.347624 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-data" (OuterVolumeSpecName: "istio-data") pod "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" (UID: "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:51:37.347768 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.347660 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" (UID: "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:51:37.349390 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.349369 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-kube-api-access-6fmhz" (OuterVolumeSpecName: "kube-api-access-6fmhz") pod "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" (UID: "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414"). InnerVolumeSpecName "kube-api-access-6fmhz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:51:37.349390 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.349369 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" (UID: "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:51:37.349390 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.349375 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" (UID: "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:51:37.349606 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.349559 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-token" (OuterVolumeSpecName: "istio-token") pod "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" (UID: "ab7e8831-cd3e-49dc-8c8c-a838d4dc6414"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:51:37.381336 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.381307 2574 generic.go:356] "Generic (PLEG): container finished" podID="ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" containerID="cb5ba09b1a6356a0a592f32b241e88a61d9b38b26dafc65f180fa71e57ffd3a0" exitCode=0 Jul 08 18:51:37.381486 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.381352 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" Jul 08 18:51:37.381486 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.381386 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" event={"ID":"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414","Type":"ContainerDied","Data":"cb5ba09b1a6356a0a592f32b241e88a61d9b38b26dafc65f180fa71e57ffd3a0"} Jul 08 18:51:37.381486 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.381451 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj" event={"ID":"ab7e8831-cd3e-49dc-8c8c-a838d4dc6414","Type":"ContainerDied","Data":"a0035f91d98193ab4114f5ffe2af509f2c4fa72a7043644461ab5c2984b2eddd"} Jul 08 18:51:37.381486 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.381481 2574 scope.go:117] "RemoveContainer" containerID="cb5ba09b1a6356a0a592f32b241e88a61d9b38b26dafc65f180fa71e57ffd3a0" Jul 08 18:51:37.390524 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.390509 2574 scope.go:117] "RemoveContainer" containerID="24f0f6fa470d783a3d4ed450f3e4e672508d70e275d091c6392c2106963244ee" Jul 08 18:51:37.398553 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.398537 2574 scope.go:117] "RemoveContainer" containerID="cb5ba09b1a6356a0a592f32b241e88a61d9b38b26dafc65f180fa71e57ffd3a0" Jul 08 18:51:37.398757 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:51:37.398741 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb5ba09b1a6356a0a592f32b241e88a61d9b38b26dafc65f180fa71e57ffd3a0\": container with ID starting with cb5ba09b1a6356a0a592f32b241e88a61d9b38b26dafc65f180fa71e57ffd3a0 not found: ID does not exist" containerID="cb5ba09b1a6356a0a592f32b241e88a61d9b38b26dafc65f180fa71e57ffd3a0" Jul 08 18:51:37.398814 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.398765 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb5ba09b1a6356a0a592f32b241e88a61d9b38b26dafc65f180fa71e57ffd3a0"} err="failed to get container status \"cb5ba09b1a6356a0a592f32b241e88a61d9b38b26dafc65f180fa71e57ffd3a0\": rpc error: code = NotFound desc = could not find container \"cb5ba09b1a6356a0a592f32b241e88a61d9b38b26dafc65f180fa71e57ffd3a0\": container with ID starting with cb5ba09b1a6356a0a592f32b241e88a61d9b38b26dafc65f180fa71e57ffd3a0 not found: ID does not exist" Jul 08 18:51:37.398814 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.398781 2574 scope.go:117] "RemoveContainer" containerID="24f0f6fa470d783a3d4ed450f3e4e672508d70e275d091c6392c2106963244ee" Jul 08 18:51:37.399067 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:51:37.399052 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24f0f6fa470d783a3d4ed450f3e4e672508d70e275d091c6392c2106963244ee\": container with ID starting with 24f0f6fa470d783a3d4ed450f3e4e672508d70e275d091c6392c2106963244ee not found: ID does not exist" containerID="24f0f6fa470d783a3d4ed450f3e4e672508d70e275d091c6392c2106963244ee" Jul 08 18:51:37.399117 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.399070 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24f0f6fa470d783a3d4ed450f3e4e672508d70e275d091c6392c2106963244ee"} err="failed to get container status \"24f0f6fa470d783a3d4ed450f3e4e672508d70e275d091c6392c2106963244ee\": rpc error: code = NotFound desc = could not find container \"24f0f6fa470d783a3d4ed450f3e4e672508d70e275d091c6392c2106963244ee\": container with ID starting with 24f0f6fa470d783a3d4ed450f3e4e672508d70e275d091c6392c2106963244ee not found: ID does not exist" Jul 08 18:51:37.403396 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.403371 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj"] Jul 08 18:51:37.407550 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.407527 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f330a831-openshift-default-64bdb8b875-p56vj"] Jul 08 18:51:37.447539 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.447514 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fmhz\" (UniqueName: \"kubernetes.io/projected/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-kube-api-access-6fmhz\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:51:37.447539 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.447540 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-envoy\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:51:37.447700 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.447555 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istiod-ca-cert\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:51:37.447700 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.447566 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-data\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:51:37.447700 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.447576 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-token\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:51:37.447700 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.447587 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414-istio-podinfo\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:51:37.771014 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:37.770935 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" path="/var/lib/kubelet/pods/ab7e8831-cd3e-49dc-8c8c-a838d4dc6414/volumes" Jul 08 18:51:41.800585 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:41.800552 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:41.858583 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:41.858532 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:42.490990 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:42.490947 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:48.382608 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:48.382572 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:48.454524 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:48.454485 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:48.537387 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:48.537358 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:48.635103 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:48.635004 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:59.120542 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:59.120495 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:51:59.472751 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:51:59.472643 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:07.004402 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:07.004359 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:20.471330 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:20.471235 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:26.127979 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.127934 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:26.228472 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.228437 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:26.494779 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.494689 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:26.791091 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.791013 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk"] Jul 08 18:52:26.791394 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.791381 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" containerName="istio-proxy" Jul 08 18:52:26.791445 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.791398 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" containerName="istio-proxy" Jul 08 18:52:26.791445 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.791414 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d302da-c5eb-48af-9079-191af30f4d7e" containerName="maas-api" Jul 08 18:52:26.791445 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.791420 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d302da-c5eb-48af-9079-191af30f4d7e" containerName="maas-api" Jul 08 18:52:26.791445 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.791440 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" containerName="istio-proxy" Jul 08 18:52:26.791445 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.791445 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" containerName="istio-proxy" Jul 08 18:52:26.791598 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.791501 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d302da-c5eb-48af-9079-191af30f4d7e" containerName="maas-api" Jul 08 18:52:26.791598 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.791509 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" containerName="istio-proxy" Jul 08 18:52:26.791598 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.791519 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab7e8831-cd3e-49dc-8c8c-a838d4dc6414" containerName="istio-proxy" Jul 08 18:52:26.794592 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.794571 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:26.797303 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.797281 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-200f0c-openshift-default-dockercfg-f5zct\"" Jul 08 18:52:26.815314 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.815283 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk"] Jul 08 18:52:26.819232 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.819203 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:26.906728 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.906690 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn44z\" (UniqueName: \"kubernetes.io/projected/2c3513e8-1fdf-4d4e-bd70-408277406429-kube-api-access-wn44z\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:26.906907 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.906745 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c3513e8-1fdf-4d4e-bd70-408277406429-istiod-ca-cert\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:26.906907 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.906773 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-data\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:26.906907 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.906795 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-envoy\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:26.906907 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.906815 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-podinfo\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:26.906907 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.906835 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-credential-socket\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:26.907178 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.906941 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-workload-certs\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:26.907178 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.906992 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-workload-socket\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:26.907178 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.907018 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-token\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:26.991221 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:26.991185 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:27.008420 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.008392 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-podinfo\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.008563 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.008428 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-credential-socket\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.008563 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.008454 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-workload-certs\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.008563 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.008478 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-workload-socket\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.008563 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.008500 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-token\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.008563 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.008553 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn44z\" (UniqueName: \"kubernetes.io/projected/2c3513e8-1fdf-4d4e-bd70-408277406429-kube-api-access-wn44z\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.008821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.008605 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c3513e8-1fdf-4d4e-bd70-408277406429-istiod-ca-cert\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.008821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.008640 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-data\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.008821 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.008674 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-envoy\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.009096 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.008841 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-credential-socket\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.009096 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.008927 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-workload-certs\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.009205 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.009104 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-workload-socket\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.009262 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.009204 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-data\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.009480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.009464 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c3513e8-1fdf-4d4e-bd70-408277406429-istiod-ca-cert\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.011002 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.010983 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-envoy\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.011242 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.011226 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-podinfo\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.016081 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.016060 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-token\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.016313 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.016295 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn44z\" (UniqueName: \"kubernetes.io/projected/2c3513e8-1fdf-4d4e-bd70-408277406429-kube-api-access-wn44z\") pod \"e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.108385 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.108347 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:27.243017 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.242987 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk"] Jul 08 18:52:27.245069 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:52:27.245042 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c3513e8_1fdf_4d4e_bd70_408277406429.slice/crio-74fe548aaf157cb7ed6a67110ab08622737549e4763cebd3ccfe2bc7b734e4bd WatchSource:0}: Error finding container 74fe548aaf157cb7ed6a67110ab08622737549e4763cebd3ccfe2bc7b734e4bd: Status 404 returned error can't find the container with id 74fe548aaf157cb7ed6a67110ab08622737549e4763cebd3ccfe2bc7b734e4bd Jul 08 18:52:27.247548 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.247514 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:52:27.247638 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.247579 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:52:27.247638 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.247613 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:52:27.607675 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.607632 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" event={"ID":"2c3513e8-1fdf-4d4e-bd70-408277406429","Type":"ContainerStarted","Data":"b06b66b026bfec540ebe19e58bcc3abaa9ddb95130b597bfe40aa279390907d8"} Jul 08 18:52:27.607889 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.607735 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" event={"ID":"2c3513e8-1fdf-4d4e-bd70-408277406429","Type":"ContainerStarted","Data":"74fe548aaf157cb7ed6a67110ab08622737549e4763cebd3ccfe2bc7b734e4bd"} Jul 08 18:52:27.630353 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:27.630306 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" podStartSLOduration=1.630292411 podStartE2EDuration="1.630292411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:52:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:52:27.627656756 +0000 UTC m=+2068.378966230" watchObservedRunningTime="2026-07-08 18:52:27.630292411 +0000 UTC m=+2068.381601945" Jul 08 18:52:28.109384 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:28.109348 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:28.114341 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:28.114316 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:28.613188 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:28.613157 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:28.614298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:28.614278 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:29.577018 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:29.576978 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:29.665584 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:29.665557 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:29.733229 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:29.733202 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:30.139165 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:30.139129 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:30.241385 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:30.241355 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:30.338536 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:30.338507 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:34.127223 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.127181 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:34.459872 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.459766 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:34.754442 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.754353 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6"] Jul 08 18:52:34.758286 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.758262 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.760834 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.760814 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-095308-openshift-default-dockercfg-zkl5t\"" Jul 08 18:52:34.775264 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.775230 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6"] Jul 08 18:52:34.781647 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.781612 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:34.884236 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.884206 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-credential-socket\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.884445 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.884296 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-envoy\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.884522 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.884444 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-token\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.884522 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.884515 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-podinfo\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.884633 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.884558 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pxnd\" (UniqueName: \"kubernetes.io/projected/1ee71116-7309-4e50-b795-2cdfad57d2be-kube-api-access-5pxnd\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.884633 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.884584 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-workload-socket\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.884753 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.884636 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ee71116-7309-4e50-b795-2cdfad57d2be-istiod-ca-cert\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.884753 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.884727 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-workload-certs\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.884890 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.884762 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-data\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.985742 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.985692 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-envoy\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.985986 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.985761 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-token\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.985986 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.985805 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-podinfo\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.985986 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.985841 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pxnd\" (UniqueName: \"kubernetes.io/projected/1ee71116-7309-4e50-b795-2cdfad57d2be-kube-api-access-5pxnd\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.985986 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.985897 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-workload-socket\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.985986 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.985931 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ee71116-7309-4e50-b795-2cdfad57d2be-istiod-ca-cert\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.985986 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.985983 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-workload-certs\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.986311 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.986014 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-data\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.986311 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.986047 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-credential-socket\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.986409 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.986317 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-workload-socket\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.986528 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.986507 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-credential-socket\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.986595 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.986520 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-data\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.986595 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.986537 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-workload-certs\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.986732 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.986715 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ee71116-7309-4e50-b795-2cdfad57d2be-istiod-ca-cert\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.988261 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.988242 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-envoy\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.988519 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.988502 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-podinfo\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.993699 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.993677 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-token\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:34.993922 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:34.993902 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pxnd\" (UniqueName: \"kubernetes.io/projected/1ee71116-7309-4e50-b795-2cdfad57d2be-kube-api-access-5pxnd\") pod \"e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:35.069088 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:35.069045 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:35.088386 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:35.088360 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:35.213212 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:35.213191 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6"] Jul 08 18:52:35.215664 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:52:35.215635 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ee71116_7309_4e50_b795_2cdfad57d2be.slice/crio-c06da94313c90fb67cd2d2baf63a8da2d3ce40b89bb852653405fe3297ebfd16 WatchSource:0}: Error finding container c06da94313c90fb67cd2d2baf63a8da2d3ce40b89bb852653405fe3297ebfd16: Status 404 returned error can't find the container with id c06da94313c90fb67cd2d2baf63a8da2d3ce40b89bb852653405fe3297ebfd16 Jul 08 18:52:35.217628 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:35.217595 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:52:35.217719 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:35.217659 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:52:35.217719 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:35.217692 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:52:35.646172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:35.646122 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" event={"ID":"1ee71116-7309-4e50-b795-2cdfad57d2be","Type":"ContainerStarted","Data":"a829fc140f84048cc4a2a7d1a280e61fabc7f015857e2c25f7b048fc60d4a948"} Jul 08 18:52:35.646370 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:35.646183 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" event={"ID":"1ee71116-7309-4e50-b795-2cdfad57d2be","Type":"ContainerStarted","Data":"c06da94313c90fb67cd2d2baf63a8da2d3ce40b89bb852653405fe3297ebfd16"} Jul 08 18:52:35.666920 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:35.666874 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" podStartSLOduration=1.666838028 podStartE2EDuration="1.666838028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:52:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:52:35.664308096 +0000 UTC m=+2076.415617597" watchObservedRunningTime="2026-07-08 18:52:35.666838028 +0000 UTC m=+2076.418147502" Jul 08 18:52:36.070233 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:36.070206 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:36.075160 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:36.075137 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:36.650802 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:36.650767 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:36.651869 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:36.651832 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:37.662173 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:37.662138 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:37.759482 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:37.759451 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:37.874005 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:37.873968 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp"] Jul 08 18:52:37.878085 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:37.878059 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" Jul 08 18:52:37.880569 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:37.880544 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-095308-serving-cert\"" Jul 08 18:52:37.880709 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:37.880629 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rlgwv\"" Jul 08 18:52:37.886760 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:37.886735 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp"] Jul 08 18:52:37.906544 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:37.906518 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:38.017223 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:38.017125 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b626b74b-d472-4882-86a2-eea51df3c5e2-maas-api-tls\") pod \"maas-api-e2e-inf-b-095308-584d655589-xmrxp\" (UID: \"b626b74b-d472-4882-86a2-eea51df3c5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" Jul 08 18:52:38.017387 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:38.017259 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7g2x\" (UniqueName: \"kubernetes.io/projected/b626b74b-d472-4882-86a2-eea51df3c5e2-kube-api-access-p7g2x\") pod \"maas-api-e2e-inf-b-095308-584d655589-xmrxp\" (UID: \"b626b74b-d472-4882-86a2-eea51df3c5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" Jul 08 18:52:38.017387 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:38.017288 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b626b74b-d472-4882-86a2-eea51df3c5e2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-095308-584d655589-xmrxp\" (UID: \"b626b74b-d472-4882-86a2-eea51df3c5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" Jul 08 18:52:38.115631 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:38.115595 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:38.117946 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:38.117927 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b626b74b-d472-4882-86a2-eea51df3c5e2-maas-api-tls\") pod \"maas-api-e2e-inf-b-095308-584d655589-xmrxp\" (UID: \"b626b74b-d472-4882-86a2-eea51df3c5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" Jul 08 18:52:38.118041 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:38.117993 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7g2x\" (UniqueName: \"kubernetes.io/projected/b626b74b-d472-4882-86a2-eea51df3c5e2-kube-api-access-p7g2x\") pod \"maas-api-e2e-inf-b-095308-584d655589-xmrxp\" (UID: \"b626b74b-d472-4882-86a2-eea51df3c5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" Jul 08 18:52:38.118041 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:38.118014 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b626b74b-d472-4882-86a2-eea51df3c5e2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-095308-584d655589-xmrxp\" (UID: \"b626b74b-d472-4882-86a2-eea51df3c5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" Jul 08 18:52:38.118579 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:38.118562 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b626b74b-d472-4882-86a2-eea51df3c5e2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-095308-584d655589-xmrxp\" (UID: \"b626b74b-d472-4882-86a2-eea51df3c5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" Jul 08 18:52:38.120580 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:38.120557 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b626b74b-d472-4882-86a2-eea51df3c5e2-maas-api-tls\") pod \"maas-api-e2e-inf-b-095308-584d655589-xmrxp\" (UID: \"b626b74b-d472-4882-86a2-eea51df3c5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" Jul 08 18:52:38.126581 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:38.126560 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7g2x\" (UniqueName: \"kubernetes.io/projected/b626b74b-d472-4882-86a2-eea51df3c5e2-kube-api-access-p7g2x\") pod \"maas-api-e2e-inf-b-095308-584d655589-xmrxp\" (UID: \"b626b74b-d472-4882-86a2-eea51df3c5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" Jul 08 18:52:38.193772 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:38.193733 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" Jul 08 18:52:38.338890 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:38.338838 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp"] Jul 08 18:52:38.341468 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:52:38.341443 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb626b74b_d472_4882_86a2_eea51df3c5e2.slice/crio-41212422b1cbd5ba2530f6c9d10db16e4f56eda3e7003d317e6ef9597263d775 WatchSource:0}: Error finding container 41212422b1cbd5ba2530f6c9d10db16e4f56eda3e7003d317e6ef9597263d775: Status 404 returned error can't find the container with id 41212422b1cbd5ba2530f6c9d10db16e4f56eda3e7003d317e6ef9597263d775 Jul 08 18:52:38.342975 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:38.342956 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:52:38.406376 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:38.406344 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:38.505212 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:38.505165 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:38.662285 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:38.662242 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" event={"ID":"b626b74b-d472-4882-86a2-eea51df3c5e2","Type":"ContainerStarted","Data":"41212422b1cbd5ba2530f6c9d10db16e4f56eda3e7003d317e6ef9597263d775"} Jul 08 18:52:39.668461 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:39.668423 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" event={"ID":"b626b74b-d472-4882-86a2-eea51df3c5e2","Type":"ContainerStarted","Data":"764da5ccd488638d8cb9d15dce031a4409c1bac3cdd32354d0918cce4e5ddd3f"} Jul 08 18:52:39.668966 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:39.668557 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" Jul 08 18:52:39.690464 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:39.690396 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" podStartSLOduration=2.167880454 podStartE2EDuration="2.690375309s" totalImagesPullingTime="522.494855ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:52:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:52:39.687759616 +0000 UTC m=+2080.439069091" watchObservedRunningTime="2026-07-08 18:52:39.690375309 +0000 UTC m=+2080.441684785" Jul 08 18:52:42.174368 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:42.174332 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:44.655077 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:44.655031 2574 patch_prober.go:28] interesting pod/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 18:52:44.655563 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:44.655117 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" podUID="2c3513e8-1fdf-4d4e-bd70-408277406429" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 18:52:45.679072 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:45.679045 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" Jul 08 18:52:46.708454 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:46.708343 2574 generic.go:356] "Generic (PLEG): container finished" podID="2c3513e8-1fdf-4d4e-bd70-408277406429" containerID="b06b66b026bfec540ebe19e58bcc3abaa9ddb95130b597bfe40aa279390907d8" exitCode=0 Jul 08 18:52:46.708454 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:46.708433 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" event={"ID":"2c3513e8-1fdf-4d4e-bd70-408277406429","Type":"ContainerDied","Data":"b06b66b026bfec540ebe19e58bcc3abaa9ddb95130b597bfe40aa279390907d8"} Jul 08 18:52:46.709059 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:46.708975 2574 scope.go:117] "RemoveContainer" containerID="b06b66b026bfec540ebe19e58bcc3abaa9ddb95130b597bfe40aa279390907d8" Jul 08 18:52:46.716537 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:46.711238 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:52:46.716537 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:46.711398 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:52:46.716537 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:46.711497 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:52:47.720827 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:47.720791 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" event={"ID":"2c3513e8-1fdf-4d4e-bd70-408277406429","Type":"ContainerStarted","Data":"e6d95e3183568f07941e56859969ada2a99941783c4379b3fd7f15cc9ecab288"} Jul 08 18:52:48.109355 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:48.109315 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:48.109759 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:48.109670 2574 patch_prober.go:28] interesting pod/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 08 18:52:48.109987 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:48.109959 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" podUID="2c3513e8-1fdf-4d4e-bd70-408277406429" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 08 18:52:49.110116 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:49.109628 2574 patch_prober.go:28] interesting pod/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 08 18:52:49.110116 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:49.109695 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" podUID="2c3513e8-1fdf-4d4e-bd70-408277406429" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 08 18:52:50.108902 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:50.108862 2574 patch_prober.go:28] interesting pod/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 08 18:52:50.109582 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:50.109186 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" podUID="2c3513e8-1fdf-4d4e-bd70-408277406429" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 08 18:52:50.825216 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:50.825180 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:50.931190 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:50.931155 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:51.109484 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:51.109395 2574 patch_prober.go:28] interesting pod/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 08 18:52:51.109484 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:51.109462 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" podUID="2c3513e8-1fdf-4d4e-bd70-408277406429" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 08 18:52:52.112290 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:52.112250 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:52.112809 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:52.112795 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:52.113608 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:52.113590 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:52:52.694000 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:52.693962 2574 patch_prober.go:28] interesting pod/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 18:52:52.694168 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:52.694017 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" podUID="1ee71116-7309-4e50-b795-2cdfad57d2be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 18:52:53.760128 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:53.760019 2574 generic.go:356] "Generic (PLEG): container finished" podID="1ee71116-7309-4e50-b795-2cdfad57d2be" containerID="a829fc140f84048cc4a2a7d1a280e61fabc7f015857e2c25f7b048fc60d4a948" exitCode=0 Jul 08 18:52:53.760128 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:53.760099 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" event={"ID":"1ee71116-7309-4e50-b795-2cdfad57d2be","Type":"ContainerDied","Data":"a829fc140f84048cc4a2a7d1a280e61fabc7f015857e2c25f7b048fc60d4a948"} Jul 08 18:52:53.760704 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:53.760565 2574 scope.go:117] "RemoveContainer" containerID="a829fc140f84048cc4a2a7d1a280e61fabc7f015857e2c25f7b048fc60d4a948" Jul 08 18:52:53.761582 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:53.761554 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:52:53.761648 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:53.761628 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:52:53.761687 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:53.761660 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:52:54.768894 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:54.768838 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" event={"ID":"1ee71116-7309-4e50-b795-2cdfad57d2be","Type":"ContainerStarted","Data":"d73642bf00e5a4d31a4e5303fd60915b1ca5d780ecba0f37ca24e853f2bd6c9c"} Jul 08 18:52:55.070006 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:55.069982 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:55.070173 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:55.070151 2574 patch_prober.go:28] interesting pod/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 08 18:52:55.070219 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:55.070197 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" podUID="1ee71116-7309-4e50-b795-2cdfad57d2be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 08 18:52:56.070101 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:56.070065 2574 patch_prober.go:28] interesting pod/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 08 18:52:56.070512 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:56.070121 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" podUID="1ee71116-7309-4e50-b795-2cdfad57d2be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 08 18:52:57.069391 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:57.069357 2574 patch_prober.go:28] interesting pod/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 08 18:52:57.069640 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:57.069410 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" podUID="1ee71116-7309-4e50-b795-2cdfad57d2be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 08 18:52:58.070230 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:58.070191 2574 patch_prober.go:28] interesting pod/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 08 18:52:58.070642 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:58.070258 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" podUID="1ee71116-7309-4e50-b795-2cdfad57d2be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 08 18:52:59.073967 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:59.073941 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:59.074413 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:59.074311 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:59.075123 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:59.075058 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:52:59.112166 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:59.112138 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:59.212911 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:59.212882 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:52:59.849947 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:52:59.849920 2574 scope.go:117] "RemoveContainer" containerID="d3979746f84c17b9ec3631ea173fa30f7831eb422b7bfa917db92fb44945cfcf" Jul 08 18:53:06.129292 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:53:06.129254 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:53:06.229275 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:53:06.229231 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:53:10.994498 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:53:10.994465 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:53:33.894422 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:53:33.894391 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:53:56.064968 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:53:56.064883 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:53:56.975780 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:53:56.975745 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:53:57.154901 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:53:57.154870 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:53:57.574509 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:53:57.574467 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:53:57.955724 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:53:57.955619 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:53:58.055922 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:53:58.055882 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:02.648048 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:02.648009 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:18.894995 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:18.894964 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:39.615289 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:39.615250 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:39.866935 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:39.866832 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk"] Jul 08 18:54:39.867097 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:39.867061 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" podUID="2c3513e8-1fdf-4d4e-bd70-408277406429" containerName="istio-proxy" containerID="cri-o://e6d95e3183568f07941e56859969ada2a99941783c4379b3fd7f15cc9ecab288" gracePeriod=30 Jul 08 18:54:39.894836 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:39.894800 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:40.981630 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:40.981599 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:41.034108 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:41.034076 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:41.108009 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:41.107980 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp"] Jul 08 18:54:41.108228 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:41.108203 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" podUID="b626b74b-d472-4882-86a2-eea51df3c5e2" containerName="maas-api" containerID="cri-o://764da5ccd488638d8cb9d15dce031a4409c1bac3cdd32354d0918cce4e5ddd3f" gracePeriod=30 Jul 08 18:54:41.129793 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:41.129766 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:41.272975 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:41.272898 2574 generic.go:356] "Generic (PLEG): container finished" podID="b626b74b-d472-4882-86a2-eea51df3c5e2" containerID="764da5ccd488638d8cb9d15dce031a4409c1bac3cdd32354d0918cce4e5ddd3f" exitCode=0 Jul 08 18:54:41.273126 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:41.272985 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" event={"ID":"b626b74b-d472-4882-86a2-eea51df3c5e2","Type":"ContainerDied","Data":"764da5ccd488638d8cb9d15dce031a4409c1bac3cdd32354d0918cce4e5ddd3f"} Jul 08 18:54:41.354396 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:41.354369 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" Jul 08 18:54:41.530288 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:41.530192 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b626b74b-d472-4882-86a2-eea51df3c5e2-maas-api-tls\") pod \"b626b74b-d472-4882-86a2-eea51df3c5e2\" (UID: \"b626b74b-d472-4882-86a2-eea51df3c5e2\") " Jul 08 18:54:41.530455 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:41.530360 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7g2x\" (UniqueName: \"kubernetes.io/projected/b626b74b-d472-4882-86a2-eea51df3c5e2-kube-api-access-p7g2x\") pod \"b626b74b-d472-4882-86a2-eea51df3c5e2\" (UID: \"b626b74b-d472-4882-86a2-eea51df3c5e2\") " Jul 08 18:54:41.530455 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:41.530392 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b626b74b-d472-4882-86a2-eea51df3c5e2-openshift-service-ca\") pod \"b626b74b-d472-4882-86a2-eea51df3c5e2\" (UID: \"b626b74b-d472-4882-86a2-eea51df3c5e2\") " Jul 08 18:54:41.530777 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:41.530746 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b626b74b-d472-4882-86a2-eea51df3c5e2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b626b74b-d472-4882-86a2-eea51df3c5e2" (UID: "b626b74b-d472-4882-86a2-eea51df3c5e2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:54:41.532461 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:41.532440 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b626b74b-d472-4882-86a2-eea51df3c5e2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b626b74b-d472-4882-86a2-eea51df3c5e2" (UID: "b626b74b-d472-4882-86a2-eea51df3c5e2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:54:41.532610 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:41.532590 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b626b74b-d472-4882-86a2-eea51df3c5e2-kube-api-access-p7g2x" (OuterVolumeSpecName: "kube-api-access-p7g2x") pod "b626b74b-d472-4882-86a2-eea51df3c5e2" (UID: "b626b74b-d472-4882-86a2-eea51df3c5e2"). InnerVolumeSpecName "kube-api-access-p7g2x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:54:41.631602 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:41.631571 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7g2x\" (UniqueName: \"kubernetes.io/projected/b626b74b-d472-4882-86a2-eea51df3c5e2-kube-api-access-p7g2x\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:41.631602 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:41.631599 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b626b74b-d472-4882-86a2-eea51df3c5e2-openshift-service-ca\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:41.631602 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:41.631608 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b626b74b-d472-4882-86a2-eea51df3c5e2-maas-api-tls\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:42.278656 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:42.278624 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" event={"ID":"b626b74b-d472-4882-86a2-eea51df3c5e2","Type":"ContainerDied","Data":"41212422b1cbd5ba2530f6c9d10db16e4f56eda3e7003d317e6ef9597263d775"} Jul 08 18:54:42.279073 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:42.278678 2574 scope.go:117] "RemoveContainer" containerID="764da5ccd488638d8cb9d15dce031a4409c1bac3cdd32354d0918cce4e5ddd3f" Jul 08 18:54:42.279073 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:42.278637 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp" Jul 08 18:54:42.296333 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:42.296310 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp"] Jul 08 18:54:42.300949 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:42.300927 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-095308-584d655589-xmrxp"] Jul 08 18:54:43.770815 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:43.770778 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b626b74b-d472-4882-86a2-eea51df3c5e2" path="/var/lib/kubelet/pods/b626b74b-d472-4882-86a2-eea51df3c5e2/volumes" Jul 08 18:54:44.892689 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:54:44.892651 2574 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c3513e8_1fdf_4d4e_bd70_408277406429.slice/crio-e6d95e3183568f07941e56859969ada2a99941783c4379b3fd7f15cc9ecab288.scope\": RecentStats: unable to find data in memory cache]" Jul 08 18:54:44.893055 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:54:44.892716 2574 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c3513e8_1fdf_4d4e_bd70_408277406429.slice/crio-e6d95e3183568f07941e56859969ada2a99941783c4379b3fd7f15cc9ecab288.scope\": RecentStats: unable to find data in memory cache]" Jul 08 18:54:45.130250 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.130192 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:54:45.168890 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.168841 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-credential-socket\") pod \"2c3513e8-1fdf-4d4e-bd70-408277406429\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " Jul 08 18:54:45.169075 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.168925 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wn44z\" (UniqueName: \"kubernetes.io/projected/2c3513e8-1fdf-4d4e-bd70-408277406429-kube-api-access-wn44z\") pod \"2c3513e8-1fdf-4d4e-bd70-408277406429\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " Jul 08 18:54:45.169075 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.168954 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-podinfo\") pod \"2c3513e8-1fdf-4d4e-bd70-408277406429\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " Jul 08 18:54:45.169075 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.168969 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-workload-socket\") pod \"2c3513e8-1fdf-4d4e-bd70-408277406429\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " Jul 08 18:54:45.169075 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.168997 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c3513e8-1fdf-4d4e-bd70-408277406429-istiod-ca-cert\") pod \"2c3513e8-1fdf-4d4e-bd70-408277406429\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " Jul 08 18:54:45.169075 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.169060 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2c3513e8-1fdf-4d4e-bd70-408277406429" (UID: "2c3513e8-1fdf-4d4e-bd70-408277406429"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:45.169376 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.169085 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-data\") pod \"2c3513e8-1fdf-4d4e-bd70-408277406429\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " Jul 08 18:54:45.169376 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.169148 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-token\") pod \"2c3513e8-1fdf-4d4e-bd70-408277406429\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " Jul 08 18:54:45.169376 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.169265 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2c3513e8-1fdf-4d4e-bd70-408277406429" (UID: "2c3513e8-1fdf-4d4e-bd70-408277406429"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:45.169376 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.169344 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-workload-certs\") pod \"2c3513e8-1fdf-4d4e-bd70-408277406429\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " Jul 08 18:54:45.169592 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.169388 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-envoy\") pod \"2c3513e8-1fdf-4d4e-bd70-408277406429\" (UID: \"2c3513e8-1fdf-4d4e-bd70-408277406429\") " Jul 08 18:54:45.169592 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.169430 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c3513e8-1fdf-4d4e-bd70-408277406429-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2c3513e8-1fdf-4d4e-bd70-408277406429" (UID: "2c3513e8-1fdf-4d4e-bd70-408277406429"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:54:45.169700 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.169641 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2c3513e8-1fdf-4d4e-bd70-408277406429" (UID: "2c3513e8-1fdf-4d4e-bd70-408277406429"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:45.169776 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.169755 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-workload-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:45.169842 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.169777 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c3513e8-1fdf-4d4e-bd70-408277406429-istiod-ca-cert\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:45.169842 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.169791 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-workload-certs\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:45.169842 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.169803 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-credential-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:45.170107 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.170082 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-data" (OuterVolumeSpecName: "istio-data") pod "2c3513e8-1fdf-4d4e-bd70-408277406429" (UID: "2c3513e8-1fdf-4d4e-bd70-408277406429"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:45.171705 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.171679 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c3513e8-1fdf-4d4e-bd70-408277406429-kube-api-access-wn44z" (OuterVolumeSpecName: "kube-api-access-wn44z") pod "2c3513e8-1fdf-4d4e-bd70-408277406429" (UID: "2c3513e8-1fdf-4d4e-bd70-408277406429"). InnerVolumeSpecName "kube-api-access-wn44z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:54:45.171828 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.171699 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-token" (OuterVolumeSpecName: "istio-token") pod "2c3513e8-1fdf-4d4e-bd70-408277406429" (UID: "2c3513e8-1fdf-4d4e-bd70-408277406429"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:54:45.171828 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.171813 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2c3513e8-1fdf-4d4e-bd70-408277406429" (UID: "2c3513e8-1fdf-4d4e-bd70-408277406429"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:54:45.172017 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.171998 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2c3513e8-1fdf-4d4e-bd70-408277406429" (UID: "2c3513e8-1fdf-4d4e-bd70-408277406429"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:45.270281 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.270240 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-data\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:45.270281 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.270272 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-token\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:45.270281 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.270284 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-envoy\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:45.270527 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.270295 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wn44z\" (UniqueName: \"kubernetes.io/projected/2c3513e8-1fdf-4d4e-bd70-408277406429-kube-api-access-wn44z\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:45.270527 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.270308 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c3513e8-1fdf-4d4e-bd70-408277406429-istio-podinfo\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:45.296142 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.296108 2574 generic.go:356] "Generic (PLEG): container finished" podID="2c3513e8-1fdf-4d4e-bd70-408277406429" containerID="e6d95e3183568f07941e56859969ada2a99941783c4379b3fd7f15cc9ecab288" exitCode=0 Jul 08 18:54:45.296292 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.296164 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" Jul 08 18:54:45.296292 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.296188 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" event={"ID":"2c3513e8-1fdf-4d4e-bd70-408277406429","Type":"ContainerDied","Data":"e6d95e3183568f07941e56859969ada2a99941783c4379b3fd7f15cc9ecab288"} Jul 08 18:54:45.296292 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.296247 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk" event={"ID":"2c3513e8-1fdf-4d4e-bd70-408277406429","Type":"ContainerDied","Data":"74fe548aaf157cb7ed6a67110ab08622737549e4763cebd3ccfe2bc7b734e4bd"} Jul 08 18:54:45.296292 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.296269 2574 scope.go:117] "RemoveContainer" containerID="e6d95e3183568f07941e56859969ada2a99941783c4379b3fd7f15cc9ecab288" Jul 08 18:54:45.305863 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.305828 2574 scope.go:117] "RemoveContainer" containerID="b06b66b026bfec540ebe19e58bcc3abaa9ddb95130b597bfe40aa279390907d8" Jul 08 18:54:45.313629 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.313614 2574 scope.go:117] "RemoveContainer" containerID="e6d95e3183568f07941e56859969ada2a99941783c4379b3fd7f15cc9ecab288" Jul 08 18:54:45.313864 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:54:45.313832 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6d95e3183568f07941e56859969ada2a99941783c4379b3fd7f15cc9ecab288\": container with ID starting with e6d95e3183568f07941e56859969ada2a99941783c4379b3fd7f15cc9ecab288 not found: ID does not exist" containerID="e6d95e3183568f07941e56859969ada2a99941783c4379b3fd7f15cc9ecab288" Jul 08 18:54:45.314374 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.314350 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6d95e3183568f07941e56859969ada2a99941783c4379b3fd7f15cc9ecab288"} err="failed to get container status \"e6d95e3183568f07941e56859969ada2a99941783c4379b3fd7f15cc9ecab288\": rpc error: code = NotFound desc = could not find container \"e6d95e3183568f07941e56859969ada2a99941783c4379b3fd7f15cc9ecab288\": container with ID starting with e6d95e3183568f07941e56859969ada2a99941783c4379b3fd7f15cc9ecab288 not found: ID does not exist" Jul 08 18:54:45.314425 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.314375 2574 scope.go:117] "RemoveContainer" containerID="b06b66b026bfec540ebe19e58bcc3abaa9ddb95130b597bfe40aa279390907d8" Jul 08 18:54:45.314810 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:54:45.314724 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b06b66b026bfec540ebe19e58bcc3abaa9ddb95130b597bfe40aa279390907d8\": container with ID starting with b06b66b026bfec540ebe19e58bcc3abaa9ddb95130b597bfe40aa279390907d8 not found: ID does not exist" containerID="b06b66b026bfec540ebe19e58bcc3abaa9ddb95130b597bfe40aa279390907d8" Jul 08 18:54:45.314810 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.314773 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b06b66b026bfec540ebe19e58bcc3abaa9ddb95130b597bfe40aa279390907d8"} err="failed to get container status \"b06b66b026bfec540ebe19e58bcc3abaa9ddb95130b597bfe40aa279390907d8\": rpc error: code = NotFound desc = could not find container \"b06b66b026bfec540ebe19e58bcc3abaa9ddb95130b597bfe40aa279390907d8\": container with ID starting with b06b66b026bfec540ebe19e58bcc3abaa9ddb95130b597bfe40aa279390907d8 not found: ID does not exist" Jul 08 18:54:45.317346 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.317325 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk"] Jul 08 18:54:45.320948 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.320924 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-200f0c-openshift-default-9669997f-hhtrk"] Jul 08 18:54:45.771151 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:45.771119 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c3513e8-1fdf-4d4e-bd70-408277406429" path="/var/lib/kubelet/pods/2c3513e8-1fdf-4d4e-bd70-408277406429/volumes" Jul 08 18:54:46.657586 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:46.657557 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:46.723913 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:46.723830 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:46.916485 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:46.916387 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6"] Jul 08 18:54:46.916653 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:46.916604 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" podUID="1ee71116-7309-4e50-b795-2cdfad57d2be" containerName="istio-proxy" containerID="cri-o://d73642bf00e5a4d31a4e5303fd60915b1ca5d780ecba0f37ca24e853f2bd6c9c" gracePeriod=30 Jul 08 18:54:46.930744 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:46.930716 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:47.550113 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.550077 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:47.883418 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.883376 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r"] Jul 08 18:54:47.883928 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.883909 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c3513e8-1fdf-4d4e-bd70-408277406429" containerName="istio-proxy" Jul 08 18:54:47.884008 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.883931 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c3513e8-1fdf-4d4e-bd70-408277406429" containerName="istio-proxy" Jul 08 18:54:47.884008 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.883956 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b626b74b-d472-4882-86a2-eea51df3c5e2" containerName="maas-api" Jul 08 18:54:47.884008 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.883964 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b626b74b-d472-4882-86a2-eea51df3c5e2" containerName="maas-api" Jul 08 18:54:47.884156 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.884076 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b626b74b-d472-4882-86a2-eea51df3c5e2" containerName="maas-api" Jul 08 18:54:47.884156 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.884088 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c3513e8-1fdf-4d4e-bd70-408277406429" containerName="istio-proxy" Jul 08 18:54:47.884156 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.884098 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c3513e8-1fdf-4d4e-bd70-408277406429" containerName="istio-proxy" Jul 08 18:54:47.888653 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.888633 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:47.891926 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.891907 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-c7d624-openshift-default-dockercfg-tkgrc\"" Jul 08 18:54:47.901050 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.901028 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r"] Jul 08 18:54:47.910735 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.910712 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:47.989977 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.989915 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-credential-socket\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:47.989977 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.989969 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-workload-certs\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:47.990237 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.990062 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-istio-envoy\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:47.990237 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.990100 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gcxm\" (UniqueName: \"kubernetes.io/projected/3f49574e-2b14-482b-acbd-015980588f2c-kube-api-access-4gcxm\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:47.990237 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.990211 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-istio-data\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:47.990406 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.990240 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f49574e-2b14-482b-acbd-015980588f2c-istiod-ca-cert\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:47.990406 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.990305 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f49574e-2b14-482b-acbd-015980588f2c-istio-podinfo\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:47.990406 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.990338 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-workload-socket\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:47.990406 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:47.990363 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f49574e-2b14-482b-acbd-015980588f2c-istio-token\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.090986 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.090946 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-istio-data\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.090986 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.090987 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f49574e-2b14-482b-acbd-015980588f2c-istiod-ca-cert\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.091233 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.091023 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f49574e-2b14-482b-acbd-015980588f2c-istio-podinfo\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.091233 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.091049 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-workload-socket\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.091233 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.091072 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f49574e-2b14-482b-acbd-015980588f2c-istio-token\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.091471 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.091235 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-credential-socket\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.091471 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.091299 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-workload-certs\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.091471 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.091358 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-istio-envoy\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.091471 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.091386 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gcxm\" (UniqueName: \"kubernetes.io/projected/3f49574e-2b14-482b-acbd-015980588f2c-kube-api-access-4gcxm\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.091471 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.091415 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-istio-data\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.091471 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.091430 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-workload-socket\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.091766 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.091657 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-credential-socket\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.091766 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.091674 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-workload-certs\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.091837 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.091774 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f49574e-2b14-482b-acbd-015980588f2c-istiod-ca-cert\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.093696 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.093670 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-istio-envoy\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.093696 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.093689 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f49574e-2b14-482b-acbd-015980588f2c-istio-podinfo\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.098892 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.098841 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f49574e-2b14-482b-acbd-015980588f2c-istio-token\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.098974 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.098893 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gcxm\" (UniqueName: \"kubernetes.io/projected/3f49574e-2b14-482b-acbd-015980588f2c-kube-api-access-4gcxm\") pod \"e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.198717 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.198632 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:48.200342 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.200315 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:48.305752 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.305712 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:48.342646 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.342394 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r"] Jul 08 18:54:48.349143 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.349105 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:54:48.349242 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.349194 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:54:48.349283 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.349237 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:54:48.401793 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:48.401771 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:49.319512 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:49.319472 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" event={"ID":"3f49574e-2b14-482b-acbd-015980588f2c","Type":"ContainerStarted","Data":"de6729bddbf741e8ff8278ab1abef5e358839a955524267e89ece7665fb70414"} Jul 08 18:54:49.319931 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:49.319521 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" event={"ID":"3f49574e-2b14-482b-acbd-015980588f2c","Type":"ContainerStarted","Data":"6613b2616ba102616219969b45070c3000974d6d4bb1d7e6f0a94b7b1286e026"} Jul 08 18:54:49.342480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:49.342431 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" podStartSLOduration=2.342416593 podStartE2EDuration="2.342416593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:54:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:54:49.340314714 +0000 UTC m=+2210.091624211" watchObservedRunningTime="2026-07-08 18:54:49.342416593 +0000 UTC m=+2210.093726067" Jul 08 18:54:50.199813 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:50.199774 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:50.205176 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:50.205152 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:50.323908 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:50.323873 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:50.324897 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:50.324877 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:54:51.338490 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.338460 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:51.424052 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.424022 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:51.429421 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.429395 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd"] Jul 08 18:54:51.429963 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.429942 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c3513e8-1fdf-4d4e-bd70-408277406429" containerName="istio-proxy" Jul 08 18:54:51.429963 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.429964 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c3513e8-1fdf-4d4e-bd70-408277406429" containerName="istio-proxy" Jul 08 18:54:51.433104 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.433084 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" Jul 08 18:54:51.435360 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.435341 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-c7d624-serving-cert\"" Jul 08 18:54:51.435467 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.435441 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rlgwv\"" Jul 08 18:54:51.451584 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.444758 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd"] Jul 08 18:54:51.508953 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.508920 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:51.522814 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.522751 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f599c9bf-462a-4c77-a002-6b569c8e4a60-openshift-service-ca\") pod \"maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd\" (UID: \"f599c9bf-462a-4c77-a002-6b569c8e4a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" Jul 08 18:54:51.523001 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.522826 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2zt2\" (UniqueName: \"kubernetes.io/projected/f599c9bf-462a-4c77-a002-6b569c8e4a60-kube-api-access-v2zt2\") pod \"maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd\" (UID: \"f599c9bf-462a-4c77-a002-6b569c8e4a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" Jul 08 18:54:51.523001 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.522924 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f599c9bf-462a-4c77-a002-6b569c8e4a60-maas-api-tls\") pod \"maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd\" (UID: \"f599c9bf-462a-4c77-a002-6b569c8e4a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" Jul 08 18:54:51.623820 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.623733 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f599c9bf-462a-4c77-a002-6b569c8e4a60-maas-api-tls\") pod \"maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd\" (UID: \"f599c9bf-462a-4c77-a002-6b569c8e4a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" Jul 08 18:54:51.624022 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.623824 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f599c9bf-462a-4c77-a002-6b569c8e4a60-openshift-service-ca\") pod \"maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd\" (UID: \"f599c9bf-462a-4c77-a002-6b569c8e4a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" Jul 08 18:54:51.624022 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.623900 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2zt2\" (UniqueName: \"kubernetes.io/projected/f599c9bf-462a-4c77-a002-6b569c8e4a60-kube-api-access-v2zt2\") pod \"maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd\" (UID: \"f599c9bf-462a-4c77-a002-6b569c8e4a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" Jul 08 18:54:51.624022 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:54:51.623906 2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-serving-cert: secret "maas-api-e2e-api-a-c7d624-serving-cert" not found Jul 08 18:54:51.624022 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:54:51.623970 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f599c9bf-462a-4c77-a002-6b569c8e4a60-maas-api-tls podName:f599c9bf-462a-4c77-a002-6b569c8e4a60 nodeName:}" failed. No retries permitted until 2026-07-08 18:54:52.123953323 +0000 UTC m=+2212.875262775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f599c9bf-462a-4c77-a002-6b569c8e4a60-maas-api-tls") pod "maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" (UID: "f599c9bf-462a-4c77-a002-6b569c8e4a60") : secret "maas-api-e2e-api-a-c7d624-serving-cert" not found Jul 08 18:54:51.624521 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.624497 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f599c9bf-462a-4c77-a002-6b569c8e4a60-openshift-service-ca\") pod \"maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd\" (UID: \"f599c9bf-462a-4c77-a002-6b569c8e4a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" Jul 08 18:54:51.632530 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.632500 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2zt2\" (UniqueName: \"kubernetes.io/projected/f599c9bf-462a-4c77-a002-6b569c8e4a60-kube-api-access-v2zt2\") pod \"maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd\" (UID: \"f599c9bf-462a-4c77-a002-6b569c8e4a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" Jul 08 18:54:51.714310 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.714275 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:51.978456 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:51.978421 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:52.079774 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.079748 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:52.129545 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.129505 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f599c9bf-462a-4c77-a002-6b569c8e4a60-maas-api-tls\") pod \"maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd\" (UID: \"f599c9bf-462a-4c77-a002-6b569c8e4a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" Jul 08 18:54:52.132410 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.132377 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f599c9bf-462a-4c77-a002-6b569c8e4a60-maas-api-tls\") pod \"maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd\" (UID: \"f599c9bf-462a-4c77-a002-6b569c8e4a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" Jul 08 18:54:52.175389 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.175366 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:54:52.230019 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.229945 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-data\") pod \"1ee71116-7309-4e50-b795-2cdfad57d2be\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " Jul 08 18:54:52.230019 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.229977 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ee71116-7309-4e50-b795-2cdfad57d2be-istiod-ca-cert\") pod \"1ee71116-7309-4e50-b795-2cdfad57d2be\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " Jul 08 18:54:52.230019 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.230007 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-envoy\") pod \"1ee71116-7309-4e50-b795-2cdfad57d2be\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " Jul 08 18:54:52.230257 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.230024 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-token\") pod \"1ee71116-7309-4e50-b795-2cdfad57d2be\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " Jul 08 18:54:52.230257 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.230052 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-credential-socket\") pod \"1ee71116-7309-4e50-b795-2cdfad57d2be\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " Jul 08 18:54:52.230257 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.230083 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pxnd\" (UniqueName: \"kubernetes.io/projected/1ee71116-7309-4e50-b795-2cdfad57d2be-kube-api-access-5pxnd\") pod \"1ee71116-7309-4e50-b795-2cdfad57d2be\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " Jul 08 18:54:52.230257 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.230122 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-podinfo\") pod \"1ee71116-7309-4e50-b795-2cdfad57d2be\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " Jul 08 18:54:52.230257 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.230206 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-workload-certs\") pod \"1ee71116-7309-4e50-b795-2cdfad57d2be\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " Jul 08 18:54:52.230257 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.230256 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-workload-socket\") pod \"1ee71116-7309-4e50-b795-2cdfad57d2be\" (UID: \"1ee71116-7309-4e50-b795-2cdfad57d2be\") " Jul 08 18:54:52.230575 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.230367 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ee71116-7309-4e50-b795-2cdfad57d2be-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1ee71116-7309-4e50-b795-2cdfad57d2be" (UID: "1ee71116-7309-4e50-b795-2cdfad57d2be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:54:52.230634 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.230604 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ee71116-7309-4e50-b795-2cdfad57d2be-istiod-ca-cert\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:52.230634 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.230607 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1ee71116-7309-4e50-b795-2cdfad57d2be" (UID: "1ee71116-7309-4e50-b795-2cdfad57d2be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:52.230921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.230899 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-data" (OuterVolumeSpecName: "istio-data") pod "1ee71116-7309-4e50-b795-2cdfad57d2be" (UID: "1ee71116-7309-4e50-b795-2cdfad57d2be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:52.231040 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.230870 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1ee71116-7309-4e50-b795-2cdfad57d2be" (UID: "1ee71116-7309-4e50-b795-2cdfad57d2be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:52.231040 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.230974 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1ee71116-7309-4e50-b795-2cdfad57d2be" (UID: "1ee71116-7309-4e50-b795-2cdfad57d2be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:52.232298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.232278 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-token" (OuterVolumeSpecName: "istio-token") pod "1ee71116-7309-4e50-b795-2cdfad57d2be" (UID: "1ee71116-7309-4e50-b795-2cdfad57d2be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:54:52.232920 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.232895 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ee71116-7309-4e50-b795-2cdfad57d2be-kube-api-access-5pxnd" (OuterVolumeSpecName: "kube-api-access-5pxnd") pod "1ee71116-7309-4e50-b795-2cdfad57d2be" (UID: "1ee71116-7309-4e50-b795-2cdfad57d2be"). InnerVolumeSpecName "kube-api-access-5pxnd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:54:52.233008 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.232955 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1ee71116-7309-4e50-b795-2cdfad57d2be" (UID: "1ee71116-7309-4e50-b795-2cdfad57d2be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:52.233055 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.233035 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1ee71116-7309-4e50-b795-2cdfad57d2be" (UID: "1ee71116-7309-4e50-b795-2cdfad57d2be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:54:52.331136 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.331108 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-data\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:52.331136 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.331129 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-envoy\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:52.331136 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.331139 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-token\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:52.331396 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.331148 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-credential-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:52.331396 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.331158 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pxnd\" (UniqueName: \"kubernetes.io/projected/1ee71116-7309-4e50-b795-2cdfad57d2be-kube-api-access-5pxnd\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:52.331396 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.331175 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ee71116-7309-4e50-b795-2cdfad57d2be-istio-podinfo\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:52.331396 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.331183 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-workload-certs\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:52.331396 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.331190 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee71116-7309-4e50-b795-2cdfad57d2be-workload-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:54:52.333418 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.333393 2574 generic.go:356] "Generic (PLEG): container finished" podID="1ee71116-7309-4e50-b795-2cdfad57d2be" containerID="d73642bf00e5a4d31a4e5303fd60915b1ca5d780ecba0f37ca24e853f2bd6c9c" exitCode=0 Jul 08 18:54:52.333545 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.333474 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" event={"ID":"1ee71116-7309-4e50-b795-2cdfad57d2be","Type":"ContainerDied","Data":"d73642bf00e5a4d31a4e5303fd60915b1ca5d780ecba0f37ca24e853f2bd6c9c"} Jul 08 18:54:52.333545 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.333496 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" Jul 08 18:54:52.333545 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.333530 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6" event={"ID":"1ee71116-7309-4e50-b795-2cdfad57d2be","Type":"ContainerDied","Data":"c06da94313c90fb67cd2d2baf63a8da2d3ce40b89bb852653405fe3297ebfd16"} Jul 08 18:54:52.333681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.333553 2574 scope.go:117] "RemoveContainer" containerID="d73642bf00e5a4d31a4e5303fd60915b1ca5d780ecba0f37ca24e853f2bd6c9c" Jul 08 18:54:52.342967 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.342952 2574 scope.go:117] "RemoveContainer" containerID="a829fc140f84048cc4a2a7d1a280e61fabc7f015857e2c25f7b048fc60d4a948" Jul 08 18:54:52.346714 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.346697 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" Jul 08 18:54:52.350644 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.350629 2574 scope.go:117] "RemoveContainer" containerID="d73642bf00e5a4d31a4e5303fd60915b1ca5d780ecba0f37ca24e853f2bd6c9c" Jul 08 18:54:52.350932 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:54:52.350915 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d73642bf00e5a4d31a4e5303fd60915b1ca5d780ecba0f37ca24e853f2bd6c9c\": container with ID starting with d73642bf00e5a4d31a4e5303fd60915b1ca5d780ecba0f37ca24e853f2bd6c9c not found: ID does not exist" containerID="d73642bf00e5a4d31a4e5303fd60915b1ca5d780ecba0f37ca24e853f2bd6c9c" Jul 08 18:54:52.351006 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.350946 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d73642bf00e5a4d31a4e5303fd60915b1ca5d780ecba0f37ca24e853f2bd6c9c"} err="failed to get container status \"d73642bf00e5a4d31a4e5303fd60915b1ca5d780ecba0f37ca24e853f2bd6c9c\": rpc error: code = NotFound desc = could not find container \"d73642bf00e5a4d31a4e5303fd60915b1ca5d780ecba0f37ca24e853f2bd6c9c\": container with ID starting with d73642bf00e5a4d31a4e5303fd60915b1ca5d780ecba0f37ca24e853f2bd6c9c not found: ID does not exist" Jul 08 18:54:52.351006 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.350969 2574 scope.go:117] "RemoveContainer" containerID="a829fc140f84048cc4a2a7d1a280e61fabc7f015857e2c25f7b048fc60d4a948" Jul 08 18:54:52.351238 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:54:52.351220 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a829fc140f84048cc4a2a7d1a280e61fabc7f015857e2c25f7b048fc60d4a948\": container with ID starting with a829fc140f84048cc4a2a7d1a280e61fabc7f015857e2c25f7b048fc60d4a948 not found: ID does not exist" containerID="a829fc140f84048cc4a2a7d1a280e61fabc7f015857e2c25f7b048fc60d4a948" Jul 08 18:54:52.351283 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.351245 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a829fc140f84048cc4a2a7d1a280e61fabc7f015857e2c25f7b048fc60d4a948"} err="failed to get container status \"a829fc140f84048cc4a2a7d1a280e61fabc7f015857e2c25f7b048fc60d4a948\": rpc error: code = NotFound desc = could not find container \"a829fc140f84048cc4a2a7d1a280e61fabc7f015857e2c25f7b048fc60d4a948\": container with ID starting with a829fc140f84048cc4a2a7d1a280e61fabc7f015857e2c25f7b048fc60d4a948 not found: ID does not exist" Jul 08 18:54:52.355178 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.355149 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6"] Jul 08 18:54:52.359762 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.359732 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-095308-openshift-default-5cfffc9857-nhcn6"] Jul 08 18:54:52.484628 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:52.484576 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd"] Jul 08 18:54:52.486624 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:54:52.486599 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf599c9bf_462a_4c77_a002_6b569c8e4a60.slice/crio-2af79dd87511c1c2bab187c2c6930b8dfad224a91319e335321cc505391c7aa9 WatchSource:0}: Error finding container 2af79dd87511c1c2bab187c2c6930b8dfad224a91319e335321cc505391c7aa9: Status 404 returned error can't find the container with id 2af79dd87511c1c2bab187c2c6930b8dfad224a91319e335321cc505391c7aa9 Jul 08 18:54:53.340412 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:53.340376 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" event={"ID":"f599c9bf-462a-4c77-a002-6b569c8e4a60","Type":"ContainerStarted","Data":"564681bc99b22d2b9c156c0b9e23b98c1e525d107ef68c8e433d6da1f70fd4dd"} Jul 08 18:54:53.340603 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:53.340440 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" event={"ID":"f599c9bf-462a-4c77-a002-6b569c8e4a60","Type":"ContainerStarted","Data":"2af79dd87511c1c2bab187c2c6930b8dfad224a91319e335321cc505391c7aa9"} Jul 08 18:54:53.340603 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:53.340471 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" Jul 08 18:54:53.358971 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:53.358926 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" podStartSLOduration=1.813281521 podStartE2EDuration="2.358912395s" totalImagesPullingTime="545.630874ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:54:53.357598383 +0000 UTC m=+2214.108907867" watchObservedRunningTime="2026-07-08 18:54:53.358912395 +0000 UTC m=+2214.110221868" Jul 08 18:54:53.771444 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:53.771370 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ee71116-7309-4e50-b795-2cdfad57d2be" path="/var/lib/kubelet/pods/1ee71116-7309-4e50-b795-2cdfad57d2be/volumes" Jul 08 18:54:55.860464 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:55.860433 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:56.667467 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.667438 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q"] Jul 08 18:54:56.667898 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.667873 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ee71116-7309-4e50-b795-2cdfad57d2be" containerName="istio-proxy" Jul 08 18:54:56.667898 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.667899 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ee71116-7309-4e50-b795-2cdfad57d2be" containerName="istio-proxy" Jul 08 18:54:56.668119 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.668022 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ee71116-7309-4e50-b795-2cdfad57d2be" containerName="istio-proxy" Jul 08 18:54:56.671093 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.671071 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.673754 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.673735 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-bd7541-openshift-default-dockercfg-hvsmx\"" Jul 08 18:54:56.682932 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.682912 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q"] Jul 08 18:54:56.773030 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.772994 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-credential-socket\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.773030 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.773031 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-data\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.773282 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.773051 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istiod-ca-cert\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.773282 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.773206 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-workload-socket\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.773364 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.773295 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-token\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.773364 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.773341 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-workload-certs\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.773442 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.773389 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-envoy\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.773526 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.773460 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-podinfo\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.773526 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.773501 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89fbl\" (UniqueName: \"kubernetes.io/projected/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-kube-api-access-89fbl\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.875045 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.875003 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-envoy\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.875597 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.875053 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-podinfo\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.875597 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.875173 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89fbl\" (UniqueName: \"kubernetes.io/projected/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-kube-api-access-89fbl\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.875597 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.875274 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-credential-socket\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.875597 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.875303 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-data\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.875597 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.875326 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istiod-ca-cert\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.875597 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.875392 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-workload-socket\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.875597 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.875492 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-token\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.875597 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.875536 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-workload-certs\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.876254 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.875648 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-credential-socket\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.876254 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.875691 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-data\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.876254 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.876039 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-workload-certs\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.876254 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.876108 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-workload-socket\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.876254 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.876176 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istiod-ca-cert\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.877456 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.877434 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-envoy\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.877723 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.877707 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-podinfo\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.882920 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.882892 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-token\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.882920 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.882899 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89fbl\" (UniqueName: \"kubernetes.io/projected/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-kube-api-access-89fbl\") pod \"e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:56.981465 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:56.981369 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:57.118670 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:57.118642 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q"] Jul 08 18:54:57.120583 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:54:57.120546 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda728e58d_4a3c_4dfc_a229_e6c6ed9654db.slice/crio-0380d137825191f75334c7d1c6e4cf5a11187b97e46adff48866932dcb1c0bb4 WatchSource:0}: Error finding container 0380d137825191f75334c7d1c6e4cf5a11187b97e46adff48866932dcb1c0bb4: Status 404 returned error can't find the container with id 0380d137825191f75334c7d1c6e4cf5a11187b97e46adff48866932dcb1c0bb4 Jul 08 18:54:57.122804 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:57.122773 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:54:57.122895 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:57.122835 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:54:57.122895 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:57.122882 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:54:57.362648 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:57.362610 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" event={"ID":"a728e58d-4a3c-4dfc-a229-e6c6ed9654db","Type":"ContainerStarted","Data":"ae0ef1116b51783bde2769704e08e1f202dd58e8f42127073b5d516765e56b57"} Jul 08 18:54:57.362827 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:57.362672 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" event={"ID":"a728e58d-4a3c-4dfc-a229-e6c6ed9654db","Type":"ContainerStarted","Data":"0380d137825191f75334c7d1c6e4cf5a11187b97e46adff48866932dcb1c0bb4"} Jul 08 18:54:57.382082 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:57.381776 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" podStartSLOduration=1.381754602 podStartE2EDuration="1.381754602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:54:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:54:57.381533841 +0000 UTC m=+2218.132843344" watchObservedRunningTime="2026-07-08 18:54:57.381754602 +0000 UTC m=+2218.133064088" Jul 08 18:54:57.981462 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:57.981433 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:58.121683 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:58.121653 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:58.987901 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:58.987842 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:59.326724 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:59.326674 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:59.351080 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:59.351056 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" Jul 08 18:54:59.372259 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:59.372234 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:59.373323 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:59.373303 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:54:59.455495 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:59.455468 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:59.558726 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:59.558693 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:59.891240 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:59.891212 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:54:59.993047 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:54:59.993003 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:00.092827 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:00.092783 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:03.842039 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:03.842005 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:06.365031 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:06.364985 2574 patch_prober.go:28] interesting pod/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 18:55:06.365427 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:06.365061 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" podUID="3f49574e-2b14-482b-acbd-015980588f2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 18:55:07.416039 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:07.416006 2574 generic.go:356] "Generic (PLEG): container finished" podID="3f49574e-2b14-482b-acbd-015980588f2c" containerID="de6729bddbf741e8ff8278ab1abef5e358839a955524267e89ece7665fb70414" exitCode=0 Jul 08 18:55:07.416493 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:07.416082 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" event={"ID":"3f49574e-2b14-482b-acbd-015980588f2c","Type":"ContainerDied","Data":"de6729bddbf741e8ff8278ab1abef5e358839a955524267e89ece7665fb70414"} Jul 08 18:55:07.416493 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:07.416463 2574 scope.go:117] "RemoveContainer" containerID="de6729bddbf741e8ff8278ab1abef5e358839a955524267e89ece7665fb70414" Jul 08 18:55:07.417519 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:07.417489 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:07.417637 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:07.417592 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:07.417718 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:07.417640 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:08.423343 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:08.423309 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" event={"ID":"3f49574e-2b14-482b-acbd-015980588f2c","Type":"ContainerStarted","Data":"8e2d1e5f1e0c4fe4b55a90830eacb7fb170dda5e05278103bd61e75760462168"} Jul 08 18:55:09.199555 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:09.199518 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:55:09.200060 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:09.200030 2574 patch_prober.go:28] interesting pod/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 08 18:55:09.200201 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:09.200105 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" podUID="3f49574e-2b14-482b-acbd-015980588f2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 08 18:55:10.199691 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:10.199639 2574 patch_prober.go:28] interesting pod/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 08 18:55:10.200271 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:10.199719 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" podUID="3f49574e-2b14-482b-acbd-015980588f2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 08 18:55:11.199549 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:11.199509 2574 patch_prober.go:28] interesting pod/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 08 18:55:11.200021 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:11.199985 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" podUID="3f49574e-2b14-482b-acbd-015980588f2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 08 18:55:12.105295 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:12.105245 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:12.199905 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:12.199840 2574 patch_prober.go:28] interesting pod/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 08 18:55:12.200224 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:12.200193 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" podUID="3f49574e-2b14-482b-acbd-015980588f2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 08 18:55:12.523458 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:12.523365 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:12.627042 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:12.627009 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:12.723624 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:12.723585 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:12.854119 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:12.854082 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd"] Jul 08 18:55:12.854366 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:12.854342 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" podUID="f599c9bf-462a-4c77-a002-6b569c8e4a60" containerName="maas-api" containerID="cri-o://564681bc99b22d2b9c156c0b9e23b98c1e525d107ef68c8e433d6da1f70fd4dd" gracePeriod=30 Jul 08 18:55:12.882275 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:12.882247 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:13.133671 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.133648 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" Jul 08 18:55:13.205411 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.205369 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:55:13.205873 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.205651 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:55:13.206796 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.206762 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:55:13.245528 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.245499 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f599c9bf-462a-4c77-a002-6b569c8e4a60-openshift-service-ca\") pod \"f599c9bf-462a-4c77-a002-6b569c8e4a60\" (UID: \"f599c9bf-462a-4c77-a002-6b569c8e4a60\") " Jul 08 18:55:13.245729 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.245559 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f599c9bf-462a-4c77-a002-6b569c8e4a60-maas-api-tls\") pod \"f599c9bf-462a-4c77-a002-6b569c8e4a60\" (UID: \"f599c9bf-462a-4c77-a002-6b569c8e4a60\") " Jul 08 18:55:13.245729 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.245595 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2zt2\" (UniqueName: \"kubernetes.io/projected/f599c9bf-462a-4c77-a002-6b569c8e4a60-kube-api-access-v2zt2\") pod \"f599c9bf-462a-4c77-a002-6b569c8e4a60\" (UID: \"f599c9bf-462a-4c77-a002-6b569c8e4a60\") " Jul 08 18:55:13.245904 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.245879 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f599c9bf-462a-4c77-a002-6b569c8e4a60-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f599c9bf-462a-4c77-a002-6b569c8e4a60" (UID: "f599c9bf-462a-4c77-a002-6b569c8e4a60"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:55:13.246029 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.246013 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f599c9bf-462a-4c77-a002-6b569c8e4a60-openshift-service-ca\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:13.248437 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.248046 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f599c9bf-462a-4c77-a002-6b569c8e4a60-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f599c9bf-462a-4c77-a002-6b569c8e4a60" (UID: "f599c9bf-462a-4c77-a002-6b569c8e4a60"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:55:13.248437 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.248136 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f599c9bf-462a-4c77-a002-6b569c8e4a60-kube-api-access-v2zt2" (OuterVolumeSpecName: "kube-api-access-v2zt2") pod "f599c9bf-462a-4c77-a002-6b569c8e4a60" (UID: "f599c9bf-462a-4c77-a002-6b569c8e4a60"). InnerVolumeSpecName "kube-api-access-v2zt2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:55:13.346822 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.346788 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f599c9bf-462a-4c77-a002-6b569c8e4a60-maas-api-tls\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:13.346822 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.346822 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2zt2\" (UniqueName: \"kubernetes.io/projected/f599c9bf-462a-4c77-a002-6b569c8e4a60-kube-api-access-v2zt2\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:13.453326 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.453241 2574 generic.go:356] "Generic (PLEG): container finished" podID="f599c9bf-462a-4c77-a002-6b569c8e4a60" containerID="564681bc99b22d2b9c156c0b9e23b98c1e525d107ef68c8e433d6da1f70fd4dd" exitCode=0 Jul 08 18:55:13.453326 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.453302 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" Jul 08 18:55:13.453506 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.453327 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" event={"ID":"f599c9bf-462a-4c77-a002-6b569c8e4a60","Type":"ContainerDied","Data":"564681bc99b22d2b9c156c0b9e23b98c1e525d107ef68c8e433d6da1f70fd4dd"} Jul 08 18:55:13.453506 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.453388 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd" event={"ID":"f599c9bf-462a-4c77-a002-6b569c8e4a60","Type":"ContainerDied","Data":"2af79dd87511c1c2bab187c2c6930b8dfad224a91319e335321cc505391c7aa9"} Jul 08 18:55:13.453506 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.453411 2574 scope.go:117] "RemoveContainer" containerID="564681bc99b22d2b9c156c0b9e23b98c1e525d107ef68c8e433d6da1f70fd4dd" Jul 08 18:55:13.462812 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.462788 2574 scope.go:117] "RemoveContainer" containerID="564681bc99b22d2b9c156c0b9e23b98c1e525d107ef68c8e433d6da1f70fd4dd" Jul 08 18:55:13.463153 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:55:13.463131 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"564681bc99b22d2b9c156c0b9e23b98c1e525d107ef68c8e433d6da1f70fd4dd\": container with ID starting with 564681bc99b22d2b9c156c0b9e23b98c1e525d107ef68c8e433d6da1f70fd4dd not found: ID does not exist" containerID="564681bc99b22d2b9c156c0b9e23b98c1e525d107ef68c8e433d6da1f70fd4dd" Jul 08 18:55:13.463242 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.463172 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"564681bc99b22d2b9c156c0b9e23b98c1e525d107ef68c8e433d6da1f70fd4dd"} err="failed to get container status \"564681bc99b22d2b9c156c0b9e23b98c1e525d107ef68c8e433d6da1f70fd4dd\": rpc error: code = NotFound desc = could not find container \"564681bc99b22d2b9c156c0b9e23b98c1e525d107ef68c8e433d6da1f70fd4dd\": container with ID starting with 564681bc99b22d2b9c156c0b9e23b98c1e525d107ef68c8e433d6da1f70fd4dd not found: ID does not exist" Jul 08 18:55:13.475443 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.475416 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd"] Jul 08 18:55:13.480215 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.480190 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c7d624-76f7b69c9b-5snrd"] Jul 08 18:55:13.771300 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:13.771222 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f599c9bf-462a-4c77-a002-6b569c8e4a60" path="/var/lib/kubelet/pods/f599c9bf-462a-4c77-a002-6b569c8e4a60/volumes" Jul 08 18:55:15.415042 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:15.414998 2574 patch_prober.go:28] interesting pod/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 18:55:15.415576 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:15.415065 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" podUID="a728e58d-4a3c-4dfc-a229-e6c6ed9654db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 18:55:16.470763 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:16.470735 2574 generic.go:356] "Generic (PLEG): container finished" podID="a728e58d-4a3c-4dfc-a229-e6c6ed9654db" containerID="ae0ef1116b51783bde2769704e08e1f202dd58e8f42127073b5d516765e56b57" exitCode=0 Jul 08 18:55:16.471228 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:16.470807 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" event={"ID":"a728e58d-4a3c-4dfc-a229-e6c6ed9654db","Type":"ContainerDied","Data":"ae0ef1116b51783bde2769704e08e1f202dd58e8f42127073b5d516765e56b57"} Jul 08 18:55:16.471290 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:16.471239 2574 scope.go:117] "RemoveContainer" containerID="ae0ef1116b51783bde2769704e08e1f202dd58e8f42127073b5d516765e56b57" Jul 08 18:55:16.472280 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:16.472250 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:16.472414 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:16.472335 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:16.472414 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:16.472384 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:17.478530 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:17.478498 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" event={"ID":"a728e58d-4a3c-4dfc-a229-e6c6ed9654db","Type":"ContainerStarted","Data":"ea3f8a7de828f742bd12e12db539778c864d1fba4f7b75ddac6307db3f529e2a"} Jul 08 18:55:17.981611 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:17.981581 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:55:17.981792 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:17.981747 2574 patch_prober.go:28] interesting pod/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 08 18:55:17.981837 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:17.981788 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" podUID="a728e58d-4a3c-4dfc-a229-e6c6ed9654db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 08 18:55:18.387447 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:18.387335 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:18.446859 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:18.446811 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:18.639298 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:18.639203 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r"] Jul 08 18:55:18.639711 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:18.639459 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" podUID="3f49574e-2b14-482b-acbd-015980588f2c" containerName="istio-proxy" containerID="cri-o://8e2d1e5f1e0c4fe4b55a90830eacb7fb170dda5e05278103bd61e75760462168" gracePeriod=30 Jul 08 18:55:18.663247 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:18.663218 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:18.982329 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:18.982244 2574 patch_prober.go:28] interesting pod/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 08 18:55:18.982329 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:18.982297 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" podUID="a728e58d-4a3c-4dfc-a229-e6c6ed9654db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 08 18:55:19.169174 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:19.169141 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:19.538160 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:19.538123 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:19.694480 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:19.694439 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:19.796199 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:19.796059 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:19.979122 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:19.979083 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:19.982245 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:19.982218 2574 patch_prober.go:28] interesting pod/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 08 18:55:19.982366 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:19.982272 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" podUID="a728e58d-4a3c-4dfc-a229-e6c6ed9654db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 08 18:55:20.982588 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:20.982507 2574 patch_prober.go:28] interesting pod/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 08 18:55:20.982588 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:20.982561 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" podUID="a728e58d-4a3c-4dfc-a229-e6c6ed9654db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 08 18:55:21.982799 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:21.982763 2574 patch_prober.go:28] interesting pod/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 08 18:55:21.983225 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:21.982817 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" podUID="a728e58d-4a3c-4dfc-a229-e6c6ed9654db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 08 18:55:22.985945 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:22.985910 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:55:22.986408 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:22.986185 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:55:22.986789 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:22.986773 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:55:23.892083 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.892057 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:55:23.958911 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.958796 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-credential-socket\") pod \"3f49574e-2b14-482b-acbd-015980588f2c\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " Jul 08 18:55:23.958911 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.958873 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-istio-envoy\") pod \"3f49574e-2b14-482b-acbd-015980588f2c\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " Jul 08 18:55:23.959145 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.958913 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f49574e-2b14-482b-acbd-015980588f2c-istio-token\") pod \"3f49574e-2b14-482b-acbd-015980588f2c\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " Jul 08 18:55:23.959145 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.958940 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f49574e-2b14-482b-acbd-015980588f2c-istiod-ca-cert\") pod \"3f49574e-2b14-482b-acbd-015980588f2c\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " Jul 08 18:55:23.959145 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.958989 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-workload-certs\") pod \"3f49574e-2b14-482b-acbd-015980588f2c\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " Jul 08 18:55:23.959145 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.959014 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gcxm\" (UniqueName: \"kubernetes.io/projected/3f49574e-2b14-482b-acbd-015980588f2c-kube-api-access-4gcxm\") pod \"3f49574e-2b14-482b-acbd-015980588f2c\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " Jul 08 18:55:23.959145 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.959043 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f49574e-2b14-482b-acbd-015980588f2c-istio-podinfo\") pod \"3f49574e-2b14-482b-acbd-015980588f2c\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " Jul 08 18:55:23.959145 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.959086 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-istio-data\") pod \"3f49574e-2b14-482b-acbd-015980588f2c\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " Jul 08 18:55:23.959145 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.959102 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3f49574e-2b14-482b-acbd-015980588f2c" (UID: "3f49574e-2b14-482b-acbd-015980588f2c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:55:23.959145 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.959111 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-workload-socket\") pod \"3f49574e-2b14-482b-acbd-015980588f2c\" (UID: \"3f49574e-2b14-482b-acbd-015980588f2c\") " Jul 08 18:55:23.959548 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.959305 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3f49574e-2b14-482b-acbd-015980588f2c" (UID: "3f49574e-2b14-482b-acbd-015980588f2c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:55:23.959548 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.959366 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3f49574e-2b14-482b-acbd-015980588f2c" (UID: "3f49574e-2b14-482b-acbd-015980588f2c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:55:23.959548 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.959428 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f49574e-2b14-482b-acbd-015980588f2c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3f49574e-2b14-482b-acbd-015980588f2c" (UID: "3f49574e-2b14-482b-acbd-015980588f2c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:55:23.959713 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.959621 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f49574e-2b14-482b-acbd-015980588f2c-istiod-ca-cert\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:23.959713 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.959640 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-workload-certs\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:23.959713 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.959655 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-workload-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:23.959713 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.959667 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-credential-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:23.960356 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.960331 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-istio-data" (OuterVolumeSpecName: "istio-data") pod "3f49574e-2b14-482b-acbd-015980588f2c" (UID: "3f49574e-2b14-482b-acbd-015980588f2c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:55:23.961344 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.961311 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f49574e-2b14-482b-acbd-015980588f2c-kube-api-access-4gcxm" (OuterVolumeSpecName: "kube-api-access-4gcxm") pod "3f49574e-2b14-482b-acbd-015980588f2c" (UID: "3f49574e-2b14-482b-acbd-015980588f2c"). InnerVolumeSpecName "kube-api-access-4gcxm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:55:23.961344 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.961331 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f49574e-2b14-482b-acbd-015980588f2c-istio-token" (OuterVolumeSpecName: "istio-token") pod "3f49574e-2b14-482b-acbd-015980588f2c" (UID: "3f49574e-2b14-482b-acbd-015980588f2c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:55:23.961489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.961431 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3f49574e-2b14-482b-acbd-015980588f2c" (UID: "3f49574e-2b14-482b-acbd-015980588f2c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:55:23.961489 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:23.961432 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3f49574e-2b14-482b-acbd-015980588f2c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3f49574e-2b14-482b-acbd-015980588f2c" (UID: "3f49574e-2b14-482b-acbd-015980588f2c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:55:24.061272 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:24.061235 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-istio-envoy\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:24.061272 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:24.061271 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f49574e-2b14-482b-acbd-015980588f2c-istio-token\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:24.061700 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:24.061284 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gcxm\" (UniqueName: \"kubernetes.io/projected/3f49574e-2b14-482b-acbd-015980588f2c-kube-api-access-4gcxm\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:24.061700 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:24.061299 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f49574e-2b14-482b-acbd-015980588f2c-istio-podinfo\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:24.061700 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:24.061310 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f49574e-2b14-482b-acbd-015980588f2c-istio-data\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:24.518504 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:24.518471 2574 generic.go:356] "Generic (PLEG): container finished" podID="3f49574e-2b14-482b-acbd-015980588f2c" containerID="8e2d1e5f1e0c4fe4b55a90830eacb7fb170dda5e05278103bd61e75760462168" exitCode=0 Jul 08 18:55:24.518663 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:24.518522 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" event={"ID":"3f49574e-2b14-482b-acbd-015980588f2c","Type":"ContainerDied","Data":"8e2d1e5f1e0c4fe4b55a90830eacb7fb170dda5e05278103bd61e75760462168"} Jul 08 18:55:24.518663 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:24.518531 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" Jul 08 18:55:24.518663 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:24.518589 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r" event={"ID":"3f49574e-2b14-482b-acbd-015980588f2c","Type":"ContainerDied","Data":"6613b2616ba102616219969b45070c3000974d6d4bb1d7e6f0a94b7b1286e026"} Jul 08 18:55:24.518663 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:24.518623 2574 scope.go:117] "RemoveContainer" containerID="8e2d1e5f1e0c4fe4b55a90830eacb7fb170dda5e05278103bd61e75760462168" Jul 08 18:55:24.528493 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:24.528475 2574 scope.go:117] "RemoveContainer" containerID="de6729bddbf741e8ff8278ab1abef5e358839a955524267e89ece7665fb70414" Jul 08 18:55:24.536460 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:24.536440 2574 scope.go:117] "RemoveContainer" containerID="8e2d1e5f1e0c4fe4b55a90830eacb7fb170dda5e05278103bd61e75760462168" Jul 08 18:55:24.536711 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:55:24.536691 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e2d1e5f1e0c4fe4b55a90830eacb7fb170dda5e05278103bd61e75760462168\": container with ID starting with 8e2d1e5f1e0c4fe4b55a90830eacb7fb170dda5e05278103bd61e75760462168 not found: ID does not exist" containerID="8e2d1e5f1e0c4fe4b55a90830eacb7fb170dda5e05278103bd61e75760462168" Jul 08 18:55:24.536763 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:24.536720 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e2d1e5f1e0c4fe4b55a90830eacb7fb170dda5e05278103bd61e75760462168"} err="failed to get container status \"8e2d1e5f1e0c4fe4b55a90830eacb7fb170dda5e05278103bd61e75760462168\": rpc error: code = NotFound desc = could not find container \"8e2d1e5f1e0c4fe4b55a90830eacb7fb170dda5e05278103bd61e75760462168\": container with ID starting with 8e2d1e5f1e0c4fe4b55a90830eacb7fb170dda5e05278103bd61e75760462168 not found: ID does not exist" Jul 08 18:55:24.536763 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:24.536736 2574 scope.go:117] "RemoveContainer" containerID="de6729bddbf741e8ff8278ab1abef5e358839a955524267e89ece7665fb70414" Jul 08 18:55:24.537003 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:55:24.536988 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de6729bddbf741e8ff8278ab1abef5e358839a955524267e89ece7665fb70414\": container with ID starting with de6729bddbf741e8ff8278ab1abef5e358839a955524267e89ece7665fb70414 not found: ID does not exist" containerID="de6729bddbf741e8ff8278ab1abef5e358839a955524267e89ece7665fb70414" Jul 08 18:55:24.537063 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:24.537004 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de6729bddbf741e8ff8278ab1abef5e358839a955524267e89ece7665fb70414"} err="failed to get container status \"de6729bddbf741e8ff8278ab1abef5e358839a955524267e89ece7665fb70414\": rpc error: code = NotFound desc = could not find container \"de6729bddbf741e8ff8278ab1abef5e358839a955524267e89ece7665fb70414\": container with ID starting with de6729bddbf741e8ff8278ab1abef5e358839a955524267e89ece7665fb70414 not found: ID does not exist" Jul 08 18:55:24.539775 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:24.539752 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r"] Jul 08 18:55:24.544308 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:24.544287 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-c7d624-openshift-default-59b6cb646c-msw9r"] Jul 08 18:55:25.508453 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:25.508421 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:25.758509 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:25.758422 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q"] Jul 08 18:55:25.758727 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:25.758676 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" podUID="a728e58d-4a3c-4dfc-a229-e6c6ed9654db" containerName="istio-proxy" containerID="cri-o://ea3f8a7de828f742bd12e12db539778c864d1fba4f7b75ddac6307db3f529e2a" gracePeriod=30 Jul 08 18:55:25.770318 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:25.770288 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f49574e-2b14-482b-acbd-015980588f2c" path="/var/lib/kubelet/pods/3f49574e-2b14-482b-acbd-015980588f2c/volumes" Jul 08 18:55:25.916468 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:25.916434 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:25.973537 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:25.973507 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:26.342778 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.342739 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:26.622114 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.622042 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs"] Jul 08 18:55:26.622462 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.622415 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ee71116-7309-4e50-b795-2cdfad57d2be" containerName="istio-proxy" Jul 08 18:55:26.622462 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.622426 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ee71116-7309-4e50-b795-2cdfad57d2be" containerName="istio-proxy" Jul 08 18:55:26.622462 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.622435 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f49574e-2b14-482b-acbd-015980588f2c" containerName="istio-proxy" Jul 08 18:55:26.622462 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.622441 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f49574e-2b14-482b-acbd-015980588f2c" containerName="istio-proxy" Jul 08 18:55:26.622462 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.622455 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f599c9bf-462a-4c77-a002-6b569c8e4a60" containerName="maas-api" Jul 08 18:55:26.622462 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.622462 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f599c9bf-462a-4c77-a002-6b569c8e4a60" containerName="maas-api" Jul 08 18:55:26.622694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.622528 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f49574e-2b14-482b-acbd-015980588f2c" containerName="istio-proxy" Jul 08 18:55:26.622694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.622541 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f49574e-2b14-482b-acbd-015980588f2c" containerName="istio-proxy" Jul 08 18:55:26.622694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.622546 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f599c9bf-462a-4c77-a002-6b569c8e4a60" containerName="maas-api" Jul 08 18:55:26.622694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.622552 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ee71116-7309-4e50-b795-2cdfad57d2be" containerName="istio-proxy" Jul 08 18:55:26.625679 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.625658 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.628169 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.628138 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-956cea-openshift-default-dockercfg-fjpbf\"" Jul 08 18:55:26.636607 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.636584 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs"] Jul 08 18:55:26.644982 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.644958 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:26.684957 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.684927 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-workload-certs\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.685107 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.684960 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-data\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.685107 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.684992 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0764d2ee-ec8d-4a91-8f67-b8242388b672-istiod-ca-cert\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.685107 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.685014 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-token\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.685287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.685126 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-workload-socket\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.685287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.685177 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-credential-socket\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.685287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.685225 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-podinfo\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.685287 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.685274 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-envoy\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.685432 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.685297 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lnhc\" (UniqueName: \"kubernetes.io/projected/0764d2ee-ec8d-4a91-8f67-b8242388b672-kube-api-access-2lnhc\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.786214 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.786176 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lnhc\" (UniqueName: \"kubernetes.io/projected/0764d2ee-ec8d-4a91-8f67-b8242388b672-kube-api-access-2lnhc\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.786404 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.786244 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-workload-certs\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.786404 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.786273 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-data\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.786404 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.786316 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0764d2ee-ec8d-4a91-8f67-b8242388b672-istiod-ca-cert\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.786404 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.786351 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-token\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.786635 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.786413 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-workload-socket\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.786635 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.786453 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-credential-socket\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.786635 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.786495 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-podinfo\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.786635 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.786522 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-envoy\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.786872 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.786787 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-data\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.786935 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.786896 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-credential-socket\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.786991 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.786951 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-workload-socket\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.787051 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.787012 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-workload-certs\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.787220 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.787193 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0764d2ee-ec8d-4a91-8f67-b8242388b672-istiod-ca-cert\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.789060 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.789031 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-envoy\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.789138 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.789119 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-podinfo\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.794125 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.794099 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-token\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.794729 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.794707 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lnhc\" (UniqueName: \"kubernetes.io/projected/0764d2ee-ec8d-4a91-8f67-b8242388b672-kube-api-access-2lnhc\") pod \"e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.936088 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.935997 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:26.969434 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:26.969405 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:27.083279 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:27.083255 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs"] Jul 08 18:55:27.085875 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:55:27.085827 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0764d2ee_ec8d_4a91_8f67_b8242388b672.slice/crio-54c52265e278d48aa5de021a5b26be0985905a2b665260679822e22e3315bed1 WatchSource:0}: Error finding container 54c52265e278d48aa5de021a5b26be0985905a2b665260679822e22e3315bed1: Status 404 returned error can't find the container with id 54c52265e278d48aa5de021a5b26be0985905a2b665260679822e22e3315bed1 Jul 08 18:55:27.088279 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:27.088244 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:27.088388 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:27.088328 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:27.088388 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:27.088373 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:27.534877 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:27.534819 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" event={"ID":"0764d2ee-ec8d-4a91-8f67-b8242388b672","Type":"ContainerStarted","Data":"4345b3cd92be4fa7a805bd373737980397ed34d875095e8b6cd4083f684b4685"} Jul 08 18:55:27.535053 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:27.534898 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" event={"ID":"0764d2ee-ec8d-4a91-8f67-b8242388b672","Type":"ContainerStarted","Data":"54c52265e278d48aa5de021a5b26be0985905a2b665260679822e22e3315bed1"} Jul 08 18:55:27.565530 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:27.565480 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" podStartSLOduration=1.565463116 podStartE2EDuration="1.565463116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:55:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:55:27.564137984 +0000 UTC m=+2248.315447471" watchObservedRunningTime="2026-07-08 18:55:27.565463116 +0000 UTC m=+2248.316772646" Jul 08 18:55:27.936336 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:27.936300 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:28.941291 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:28.941262 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:29.530673 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:29.530631 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:29.543816 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:29.543780 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:29.544825 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:29.544808 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:29.650555 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:29.650501 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:29.744424 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:29.744389 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:30.112705 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:30.112671 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:30.369727 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:30.369644 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:30.467684 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:30.467651 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:31.013465 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.013442 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:55:31.126723 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.126682 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istiod-ca-cert\") pod \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " Jul 08 18:55:31.126723 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.126728 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-token\") pod \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " Jul 08 18:55:31.127272 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.126771 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-workload-certs\") pod \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " Jul 08 18:55:31.127272 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.126833 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-podinfo\") pod \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " Jul 08 18:55:31.127272 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.126905 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-envoy\") pod \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " Jul 08 18:55:31.127272 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.126933 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-credential-socket\") pod \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " Jul 08 18:55:31.127272 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.127001 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-workload-socket\") pod \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " Jul 08 18:55:31.127272 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.127038 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89fbl\" (UniqueName: \"kubernetes.io/projected/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-kube-api-access-89fbl\") pod \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " Jul 08 18:55:31.127272 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.127101 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-data\") pod \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\" (UID: \"a728e58d-4a3c-4dfc-a229-e6c6ed9654db\") " Jul 08 18:55:31.127272 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.127166 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a728e58d-4a3c-4dfc-a229-e6c6ed9654db" (UID: "a728e58d-4a3c-4dfc-a229-e6c6ed9654db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:55:31.127272 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.127225 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a728e58d-4a3c-4dfc-a229-e6c6ed9654db" (UID: "a728e58d-4a3c-4dfc-a229-e6c6ed9654db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:55:31.127694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.127270 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a728e58d-4a3c-4dfc-a229-e6c6ed9654db" (UID: "a728e58d-4a3c-4dfc-a229-e6c6ed9654db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:55:31.127694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.127180 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a728e58d-4a3c-4dfc-a229-e6c6ed9654db" (UID: "a728e58d-4a3c-4dfc-a229-e6c6ed9654db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:55:31.127694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.127527 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istiod-ca-cert\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:31.127694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.127551 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-workload-certs\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:31.127694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.127564 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-credential-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:31.127694 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.127579 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-workload-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:31.128140 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.128115 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-data" (OuterVolumeSpecName: "istio-data") pod "a728e58d-4a3c-4dfc-a229-e6c6ed9654db" (UID: "a728e58d-4a3c-4dfc-a229-e6c6ed9654db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:55:31.129346 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.129315 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a728e58d-4a3c-4dfc-a229-e6c6ed9654db" (UID: "a728e58d-4a3c-4dfc-a229-e6c6ed9654db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:55:31.129471 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.129439 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-token" (OuterVolumeSpecName: "istio-token") pod "a728e58d-4a3c-4dfc-a229-e6c6ed9654db" (UID: "a728e58d-4a3c-4dfc-a229-e6c6ed9654db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:55:31.129540 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.129502 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a728e58d-4a3c-4dfc-a229-e6c6ed9654db" (UID: "a728e58d-4a3c-4dfc-a229-e6c6ed9654db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:55:31.129540 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.129525 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-kube-api-access-89fbl" (OuterVolumeSpecName: "kube-api-access-89fbl") pod "a728e58d-4a3c-4dfc-a229-e6c6ed9654db" (UID: "a728e58d-4a3c-4dfc-a229-e6c6ed9654db"). InnerVolumeSpecName "kube-api-access-89fbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:55:31.228318 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.228273 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-data\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:31.228318 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.228315 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-token\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:31.228318 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.228325 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-podinfo\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:31.228318 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.228333 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-istio-envoy\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:31.228579 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.228341 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89fbl\" (UniqueName: \"kubernetes.io/projected/a728e58d-4a3c-4dfc-a229-e6c6ed9654db-kube-api-access-89fbl\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:55:31.554746 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.554654 2574 generic.go:356] "Generic (PLEG): container finished" podID="a728e58d-4a3c-4dfc-a229-e6c6ed9654db" containerID="ea3f8a7de828f742bd12e12db539778c864d1fba4f7b75ddac6307db3f529e2a" exitCode=0 Jul 08 18:55:31.554746 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.554728 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" Jul 08 18:55:31.555018 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.554752 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" event={"ID":"a728e58d-4a3c-4dfc-a229-e6c6ed9654db","Type":"ContainerDied","Data":"ea3f8a7de828f742bd12e12db539778c864d1fba4f7b75ddac6307db3f529e2a"} Jul 08 18:55:31.555018 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.554824 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q" event={"ID":"a728e58d-4a3c-4dfc-a229-e6c6ed9654db","Type":"ContainerDied","Data":"0380d137825191f75334c7d1c6e4cf5a11187b97e46adff48866932dcb1c0bb4"} Jul 08 18:55:31.555018 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.554882 2574 scope.go:117] "RemoveContainer" containerID="ea3f8a7de828f742bd12e12db539778c864d1fba4f7b75ddac6307db3f529e2a" Jul 08 18:55:31.564147 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.564130 2574 scope.go:117] "RemoveContainer" containerID="ae0ef1116b51783bde2769704e08e1f202dd58e8f42127073b5d516765e56b57" Jul 08 18:55:31.571958 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.571937 2574 scope.go:117] "RemoveContainer" containerID="ea3f8a7de828f742bd12e12db539778c864d1fba4f7b75ddac6307db3f529e2a" Jul 08 18:55:31.572182 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:55:31.572164 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea3f8a7de828f742bd12e12db539778c864d1fba4f7b75ddac6307db3f529e2a\": container with ID starting with ea3f8a7de828f742bd12e12db539778c864d1fba4f7b75ddac6307db3f529e2a not found: ID does not exist" containerID="ea3f8a7de828f742bd12e12db539778c864d1fba4f7b75ddac6307db3f529e2a" Jul 08 18:55:31.572247 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.572190 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea3f8a7de828f742bd12e12db539778c864d1fba4f7b75ddac6307db3f529e2a"} err="failed to get container status \"ea3f8a7de828f742bd12e12db539778c864d1fba4f7b75ddac6307db3f529e2a\": rpc error: code = NotFound desc = could not find container \"ea3f8a7de828f742bd12e12db539778c864d1fba4f7b75ddac6307db3f529e2a\": container with ID starting with ea3f8a7de828f742bd12e12db539778c864d1fba4f7b75ddac6307db3f529e2a not found: ID does not exist" Jul 08 18:55:31.572247 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.572205 2574 scope.go:117] "RemoveContainer" containerID="ae0ef1116b51783bde2769704e08e1f202dd58e8f42127073b5d516765e56b57" Jul 08 18:55:31.572402 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:55:31.572387 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae0ef1116b51783bde2769704e08e1f202dd58e8f42127073b5d516765e56b57\": container with ID starting with ae0ef1116b51783bde2769704e08e1f202dd58e8f42127073b5d516765e56b57 not found: ID does not exist" containerID="ae0ef1116b51783bde2769704e08e1f202dd58e8f42127073b5d516765e56b57" Jul 08 18:55:31.572442 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.572404 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae0ef1116b51783bde2769704e08e1f202dd58e8f42127073b5d516765e56b57"} err="failed to get container status \"ae0ef1116b51783bde2769704e08e1f202dd58e8f42127073b5d516765e56b57\": rpc error: code = NotFound desc = could not find container \"ae0ef1116b51783bde2769704e08e1f202dd58e8f42127073b5d516765e56b57\": container with ID starting with ae0ef1116b51783bde2769704e08e1f202dd58e8f42127073b5d516765e56b57 not found: ID does not exist" Jul 08 18:55:31.576801 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.576781 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q"] Jul 08 18:55:31.580485 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.580461 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-bd7541-openshift-default-55fd57f46c-fx97q"] Jul 08 18:55:31.771732 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:31.771698 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a728e58d-4a3c-4dfc-a229-e6c6ed9654db" path="/var/lib/kubelet/pods/a728e58d-4a3c-4dfc-a229-e6c6ed9654db/volumes" Jul 08 18:55:34.020467 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.020431 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:34.368461 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.368435 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:34.651355 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.651275 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt"] Jul 08 18:55:34.651636 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.651624 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a728e58d-4a3c-4dfc-a229-e6c6ed9654db" containerName="istio-proxy" Jul 08 18:55:34.651681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.651638 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a728e58d-4a3c-4dfc-a229-e6c6ed9654db" containerName="istio-proxy" Jul 08 18:55:34.651681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.651660 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f49574e-2b14-482b-acbd-015980588f2c" containerName="istio-proxy" Jul 08 18:55:34.651681 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.651667 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f49574e-2b14-482b-acbd-015980588f2c" containerName="istio-proxy" Jul 08 18:55:34.651832 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.651771 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a728e58d-4a3c-4dfc-a229-e6c6ed9654db" containerName="istio-proxy" Jul 08 18:55:34.651832 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.651786 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a728e58d-4a3c-4dfc-a229-e6c6ed9654db" containerName="istio-proxy" Jul 08 18:55:34.655166 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.655147 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.657497 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.657474 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-c926a5-openshift-default-dockercfg-chh67\"" Jul 08 18:55:34.667604 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.667584 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt"] Jul 08 18:55:34.678134 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.678114 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:34.761638 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.761604 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-workload-socket\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.761863 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.761664 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-credential-socket\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.761863 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.761694 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvqqh\" (UniqueName: \"kubernetes.io/projected/89588de2-47bf-40f2-8e25-adc56462f100-kube-api-access-fvqqh\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.761863 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.761729 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89588de2-47bf-40f2-8e25-adc56462f100-istio-podinfo\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.762044 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.761871 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-istio-envoy\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.762044 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.761927 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89588de2-47bf-40f2-8e25-adc56462f100-istiod-ca-cert\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.762044 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.761978 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89588de2-47bf-40f2-8e25-adc56462f100-istio-token\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.762193 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.762043 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-workload-certs\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.762193 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.762091 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-istio-data\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.863344 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.863312 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-workload-socket\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.863494 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.863352 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-credential-socket\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.863494 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.863372 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvqqh\" (UniqueName: \"kubernetes.io/projected/89588de2-47bf-40f2-8e25-adc56462f100-kube-api-access-fvqqh\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.863494 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.863405 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89588de2-47bf-40f2-8e25-adc56462f100-istio-podinfo\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.863494 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.863455 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-istio-envoy\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.863741 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.863616 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89588de2-47bf-40f2-8e25-adc56462f100-istiod-ca-cert\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.863741 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.863696 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89588de2-47bf-40f2-8e25-adc56462f100-istio-token\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.863874 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.863796 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-credential-socket\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.863874 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.863811 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-workload-certs\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.864094 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.863990 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-istio-data\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.864094 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.864045 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-workload-socket\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.864368 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.864335 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-istio-data\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.864506 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.864489 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89588de2-47bf-40f2-8e25-adc56462f100-istiod-ca-cert\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.864631 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.864614 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-workload-certs\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.865999 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.865980 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-istio-envoy\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.866093 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.866070 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89588de2-47bf-40f2-8e25-adc56462f100-istio-podinfo\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.872645 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.872613 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89588de2-47bf-40f2-8e25-adc56462f100-istio-token\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.872747 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.872708 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvqqh\" (UniqueName: \"kubernetes.io/projected/89588de2-47bf-40f2-8e25-adc56462f100-kube-api-access-fvqqh\") pod \"e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:34.953802 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.953719 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:34.965110 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:34.965079 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:35.102434 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:35.102410 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt"] Jul 08 18:55:35.104436 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:55:35.104410 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89588de2_47bf_40f2_8e25_adc56462f100.slice/crio-c059d58e13abb6bd889a15081adf164473041cae44056d4ebc4d389265abbd64 WatchSource:0}: Error finding container c059d58e13abb6bd889a15081adf164473041cae44056d4ebc4d389265abbd64: Status 404 returned error can't find the container with id c059d58e13abb6bd889a15081adf164473041cae44056d4ebc4d389265abbd64 Jul 08 18:55:35.106632 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:35.106600 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:35.106723 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:35.106659 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:35.106723 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:35.106686 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:35.576907 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:35.576869 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" event={"ID":"89588de2-47bf-40f2-8e25-adc56462f100","Type":"ContainerStarted","Data":"8c65bc206737a74712d2426d12301b5500acd45fe6819696320e08af7842599e"} Jul 08 18:55:35.577087 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:35.576923 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" event={"ID":"89588de2-47bf-40f2-8e25-adc56462f100","Type":"ContainerStarted","Data":"c059d58e13abb6bd889a15081adf164473041cae44056d4ebc4d389265abbd64"} Jul 08 18:55:35.601192 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:35.601147 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" podStartSLOduration=1.601131434 podStartE2EDuration="1.601131434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:55:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:55:35.599083965 +0000 UTC m=+2256.350393440" watchObservedRunningTime="2026-07-08 18:55:35.601131434 +0000 UTC m=+2256.352440944" Jul 08 18:55:35.965431 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:35.965346 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:36.970912 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:36.970882 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:37.560692 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:37.560660 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:37.586662 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:37.586635 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:37.587505 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:37.587486 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:37.861560 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:37.861475 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:37.965073 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:37.965040 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:38.066709 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:38.066677 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:38.428698 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:38.428664 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:38.482211 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:38.482175 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:42.082550 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:42.082513 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:45.586112 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:45.586048 2574 patch_prober.go:28] interesting pod/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 18:55:45.586784 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:45.586749 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" podUID="0764d2ee-ec8d-4a91-8f67-b8242388b672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 18:55:46.638894 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:46.638835 2574 generic.go:356] "Generic (PLEG): container finished" podID="0764d2ee-ec8d-4a91-8f67-b8242388b672" containerID="4345b3cd92be4fa7a805bd373737980397ed34d875095e8b6cd4083f684b4685" exitCode=0 Jul 08 18:55:46.639333 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:46.638909 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" event={"ID":"0764d2ee-ec8d-4a91-8f67-b8242388b672","Type":"ContainerDied","Data":"4345b3cd92be4fa7a805bd373737980397ed34d875095e8b6cd4083f684b4685"} Jul 08 18:55:46.639333 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:46.639310 2574 scope.go:117] "RemoveContainer" containerID="4345b3cd92be4fa7a805bd373737980397ed34d875095e8b6cd4083f684b4685" Jul 08 18:55:46.640331 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:46.640304 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:46.640387 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:46.640372 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:46.640422 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:46.640407 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:47.646172 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:47.646141 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" event={"ID":"0764d2ee-ec8d-4a91-8f67-b8242388b672","Type":"ContainerStarted","Data":"ac8c1f25d03d4c60ed306a6fa2acf62a7392af6e0851b1128209b3c11f96a34a"} Jul 08 18:55:47.936390 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:47.936285 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:47.936562 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:47.936412 2574 patch_prober.go:28] interesting pod/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 08 18:55:47.936562 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:47.936470 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" podUID="0764d2ee-ec8d-4a91-8f67-b8242388b672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 08 18:55:48.937798 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:48.937304 2574 patch_prober.go:28] interesting pod/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 08 18:55:48.937798 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:48.937380 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" podUID="0764d2ee-ec8d-4a91-8f67-b8242388b672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 08 18:55:49.937230 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:49.937136 2574 patch_prober.go:28] interesting pod/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 08 18:55:49.937230 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:49.937188 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" podUID="0764d2ee-ec8d-4a91-8f67-b8242388b672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 08 18:55:50.936798 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:50.936757 2574 patch_prober.go:28] interesting pod/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 08 18:55:50.937236 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:50.936830 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" podUID="0764d2ee-ec8d-4a91-8f67-b8242388b672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 08 18:55:51.071798 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:51.071758 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:51.171680 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:51.171641 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:51.937418 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:51.937376 2574 patch_prober.go:28] interesting pod/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 08 18:55:51.937820 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:51.937443 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" podUID="0764d2ee-ec8d-4a91-8f67-b8242388b672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 08 18:55:52.940000 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:52.939967 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:52.941309 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:52.940236 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:52.941309 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:52.941056 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:55:53.021956 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:53.021613 2574 patch_prober.go:28] interesting pod/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": read tcp 10.133.0.2:38280->10.133.0.43:15021: read: connection reset by peer" start-of-body= Jul 08 18:55:53.021956 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:53.021675 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" podUID="89588de2-47bf-40f2-8e25-adc56462f100" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": read tcp 10.133.0.2:38280->10.133.0.43:15021: read: connection reset by peer" Jul 08 18:55:53.679038 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:53.679009 2574 generic.go:356] "Generic (PLEG): container finished" podID="89588de2-47bf-40f2-8e25-adc56462f100" containerID="8c65bc206737a74712d2426d12301b5500acd45fe6819696320e08af7842599e" exitCode=0 Jul 08 18:55:53.679230 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:53.679088 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" event={"ID":"89588de2-47bf-40f2-8e25-adc56462f100","Type":"ContainerDied","Data":"8c65bc206737a74712d2426d12301b5500acd45fe6819696320e08af7842599e"} Jul 08 18:55:53.679502 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:53.679488 2574 scope.go:117] "RemoveContainer" containerID="8c65bc206737a74712d2426d12301b5500acd45fe6819696320e08af7842599e" Jul 08 18:55:53.680507 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:53.680481 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:53.680565 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:53.680555 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:53.680600 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:53.680588 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 18:55:54.687616 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:54.687583 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" event={"ID":"89588de2-47bf-40f2-8e25-adc56462f100","Type":"ContainerStarted","Data":"5c488b75ebb0f3f31250a01aa960a50e2455b49c3cecb33b026fdbe2975e624a"} Jul 08 18:55:54.966145 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:54.966052 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:54.966302 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:54.966286 2574 patch_prober.go:28] interesting pod/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 08 18:55:54.966347 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:54.966331 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" podUID="89588de2-47bf-40f2-8e25-adc56462f100" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 08 18:55:55.966253 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:55.966213 2574 patch_prober.go:28] interesting pod/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 08 18:55:55.966655 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:55.966273 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" podUID="89588de2-47bf-40f2-8e25-adc56462f100" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 08 18:55:56.965949 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:56.965909 2574 patch_prober.go:28] interesting pod/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 08 18:55:56.966144 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:56.965969 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" podUID="89588de2-47bf-40f2-8e25-adc56462f100" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 08 18:55:57.965715 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:57.965673 2574 patch_prober.go:28] interesting pod/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 08 18:55:57.966144 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:57.965730 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" podUID="89588de2-47bf-40f2-8e25-adc56462f100" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 08 18:55:58.929138 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:58.929102 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:55:58.969434 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:58.969402 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:58.969876 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:58.969684 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:58.970366 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:58.970351 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:55:58.993530 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:55:58.993495 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:56:49.854921 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:56:49.854885 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:57:27.123583 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:57:27.123547 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:58:47.254283 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:47.254220 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:58:47.671504 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:47.671464 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:58:47.776492 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:47.776456 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:58:47.930139 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:47.930054 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:58:53.548724 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:53.548687 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:58:53.617039 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:53.617005 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:58:53.825671 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:53.825641 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs"] Jul 08 18:58:53.825901 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:53.825868 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" podUID="0764d2ee-ec8d-4a91-8f67-b8242388b672" containerName="istio-proxy" containerID="cri-o://ac8c1f25d03d4c60ed306a6fa2acf62a7392af6e0851b1128209b3c11f96a34a" gracePeriod=30 Jul 08 18:58:53.853144 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:53.853118 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:58:54.057028 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:54.056992 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:58:54.260282 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:54.260201 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:58:54.755014 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:54.754977 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:58:54.853588 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:54.853541 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:58:54.968562 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:54.968532 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:58:59.076739 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.076716 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:58:59.167581 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.167501 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-envoy\") pod \"0764d2ee-ec8d-4a91-8f67-b8242388b672\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " Jul 08 18:58:59.167581 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.167538 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lnhc\" (UniqueName: \"kubernetes.io/projected/0764d2ee-ec8d-4a91-8f67-b8242388b672-kube-api-access-2lnhc\") pod \"0764d2ee-ec8d-4a91-8f67-b8242388b672\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " Jul 08 18:58:59.167581 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.167579 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-workload-certs\") pod \"0764d2ee-ec8d-4a91-8f67-b8242388b672\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " Jul 08 18:58:59.167897 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.167617 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-podinfo\") pod \"0764d2ee-ec8d-4a91-8f67-b8242388b672\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " Jul 08 18:58:59.167897 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.167645 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-data\") pod \"0764d2ee-ec8d-4a91-8f67-b8242388b672\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " Jul 08 18:58:59.167897 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.167705 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-token\") pod \"0764d2ee-ec8d-4a91-8f67-b8242388b672\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " Jul 08 18:58:59.167897 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.167737 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0764d2ee-ec8d-4a91-8f67-b8242388b672-istiod-ca-cert\") pod \"0764d2ee-ec8d-4a91-8f67-b8242388b672\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " Jul 08 18:58:59.167897 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.167765 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-credential-socket\") pod \"0764d2ee-ec8d-4a91-8f67-b8242388b672\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " Jul 08 18:58:59.167897 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.167871 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-workload-socket\") pod \"0764d2ee-ec8d-4a91-8f67-b8242388b672\" (UID: \"0764d2ee-ec8d-4a91-8f67-b8242388b672\") " Jul 08 18:58:59.168226 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.167918 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0764d2ee-ec8d-4a91-8f67-b8242388b672" (UID: "0764d2ee-ec8d-4a91-8f67-b8242388b672"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:58:59.168226 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.168128 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0764d2ee-ec8d-4a91-8f67-b8242388b672" (UID: "0764d2ee-ec8d-4a91-8f67-b8242388b672"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:58:59.168226 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.168207 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0764d2ee-ec8d-4a91-8f67-b8242388b672-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0764d2ee-ec8d-4a91-8f67-b8242388b672" (UID: "0764d2ee-ec8d-4a91-8f67-b8242388b672"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:58:59.168566 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.168292 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-workload-certs\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:58:59.168566 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.168311 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-credential-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:58:59.168566 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.168494 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0764d2ee-ec8d-4a91-8f67-b8242388b672" (UID: "0764d2ee-ec8d-4a91-8f67-b8242388b672"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:58:59.169337 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.169303 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-data" (OuterVolumeSpecName: "istio-data") pod "0764d2ee-ec8d-4a91-8f67-b8242388b672" (UID: "0764d2ee-ec8d-4a91-8f67-b8242388b672"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:58:59.169864 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.169829 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0764d2ee-ec8d-4a91-8f67-b8242388b672" (UID: "0764d2ee-ec8d-4a91-8f67-b8242388b672"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:58:59.170073 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.170053 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0764d2ee-ec8d-4a91-8f67-b8242388b672-kube-api-access-2lnhc" (OuterVolumeSpecName: "kube-api-access-2lnhc") pod "0764d2ee-ec8d-4a91-8f67-b8242388b672" (UID: "0764d2ee-ec8d-4a91-8f67-b8242388b672"). InnerVolumeSpecName "kube-api-access-2lnhc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:58:59.170143 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.170090 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-token" (OuterVolumeSpecName: "istio-token") pod "0764d2ee-ec8d-4a91-8f67-b8242388b672" (UID: "0764d2ee-ec8d-4a91-8f67-b8242388b672"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:58:59.170143 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.170124 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0764d2ee-ec8d-4a91-8f67-b8242388b672" (UID: "0764d2ee-ec8d-4a91-8f67-b8242388b672"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:58:59.268977 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.268941 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-token\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:58:59.268977 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.268969 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0764d2ee-ec8d-4a91-8f67-b8242388b672-istiod-ca-cert\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:58:59.268977 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.268982 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-workload-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:58:59.268977 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.268990 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-envoy\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:58:59.269258 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.268999 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lnhc\" (UniqueName: \"kubernetes.io/projected/0764d2ee-ec8d-4a91-8f67-b8242388b672-kube-api-access-2lnhc\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:58:59.269258 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.269009 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-podinfo\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:58:59.269258 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.269017 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0764d2ee-ec8d-4a91-8f67-b8242388b672-istio-data\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:58:59.583310 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.583267 2574 generic.go:356] "Generic (PLEG): container finished" podID="0764d2ee-ec8d-4a91-8f67-b8242388b672" containerID="ac8c1f25d03d4c60ed306a6fa2acf62a7392af6e0851b1128209b3c11f96a34a" exitCode=0 Jul 08 18:58:59.583310 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.583314 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" event={"ID":"0764d2ee-ec8d-4a91-8f67-b8242388b672","Type":"ContainerDied","Data":"ac8c1f25d03d4c60ed306a6fa2acf62a7392af6e0851b1128209b3c11f96a34a"} Jul 08 18:58:59.583580 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.583330 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" Jul 08 18:58:59.583580 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.583392 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs" event={"ID":"0764d2ee-ec8d-4a91-8f67-b8242388b672","Type":"ContainerDied","Data":"54c52265e278d48aa5de021a5b26be0985905a2b665260679822e22e3315bed1"} Jul 08 18:58:59.583580 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.583421 2574 scope.go:117] "RemoveContainer" containerID="ac8c1f25d03d4c60ed306a6fa2acf62a7392af6e0851b1128209b3c11f96a34a" Jul 08 18:58:59.593202 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.593174 2574 scope.go:117] "RemoveContainer" containerID="4345b3cd92be4fa7a805bd373737980397ed34d875095e8b6cd4083f684b4685" Jul 08 18:58:59.600964 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.600909 2574 scope.go:117] "RemoveContainer" containerID="ac8c1f25d03d4c60ed306a6fa2acf62a7392af6e0851b1128209b3c11f96a34a" Jul 08 18:58:59.601185 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:58:59.601169 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac8c1f25d03d4c60ed306a6fa2acf62a7392af6e0851b1128209b3c11f96a34a\": container with ID starting with ac8c1f25d03d4c60ed306a6fa2acf62a7392af6e0851b1128209b3c11f96a34a not found: ID does not exist" containerID="ac8c1f25d03d4c60ed306a6fa2acf62a7392af6e0851b1128209b3c11f96a34a" Jul 08 18:58:59.601226 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.601194 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac8c1f25d03d4c60ed306a6fa2acf62a7392af6e0851b1128209b3c11f96a34a"} err="failed to get container status \"ac8c1f25d03d4c60ed306a6fa2acf62a7392af6e0851b1128209b3c11f96a34a\": rpc error: code = NotFound desc = could not find container \"ac8c1f25d03d4c60ed306a6fa2acf62a7392af6e0851b1128209b3c11f96a34a\": container with ID starting with ac8c1f25d03d4c60ed306a6fa2acf62a7392af6e0851b1128209b3c11f96a34a not found: ID does not exist" Jul 08 18:58:59.601226 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.601211 2574 scope.go:117] "RemoveContainer" containerID="4345b3cd92be4fa7a805bd373737980397ed34d875095e8b6cd4083f684b4685" Jul 08 18:58:59.601443 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:58:59.601426 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4345b3cd92be4fa7a805bd373737980397ed34d875095e8b6cd4083f684b4685\": container with ID starting with 4345b3cd92be4fa7a805bd373737980397ed34d875095e8b6cd4083f684b4685 not found: ID does not exist" containerID="4345b3cd92be4fa7a805bd373737980397ed34d875095e8b6cd4083f684b4685" Jul 08 18:58:59.601479 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.601450 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4345b3cd92be4fa7a805bd373737980397ed34d875095e8b6cd4083f684b4685"} err="failed to get container status \"4345b3cd92be4fa7a805bd373737980397ed34d875095e8b6cd4083f684b4685\": rpc error: code = NotFound desc = could not find container \"4345b3cd92be4fa7a805bd373737980397ed34d875095e8b6cd4083f684b4685\": container with ID starting with 4345b3cd92be4fa7a805bd373737980397ed34d875095e8b6cd4083f684b4685 not found: ID does not exist" Jul 08 18:58:59.604713 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.604688 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs"] Jul 08 18:58:59.611046 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.611021 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-956cea-openshift-default-6f9bf4bcd7-tvhfs"] Jul 08 18:58:59.771328 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:58:59.771289 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0764d2ee-ec8d-4a91-8f67-b8242388b672" path="/var/lib/kubelet/pods/0764d2ee-ec8d-4a91-8f67-b8242388b672/volumes" Jul 08 18:59:00.523246 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:00.523210 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:59:00.623158 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:00.623129 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:59:00.783664 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:00.783585 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt"] Jul 08 18:59:00.783813 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:00.783786 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" podUID="89588de2-47bf-40f2-8e25-adc56462f100" containerName="istio-proxy" containerID="cri-o://5c488b75ebb0f3f31250a01aa960a50e2455b49c3cecb33b026fdbe2975e624a" gracePeriod=30 Jul 08 18:59:00.805452 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:00.805417 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:59:06.039878 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.039831 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:59:06.133827 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.133788 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89588de2-47bf-40f2-8e25-adc56462f100-istiod-ca-cert\") pod \"89588de2-47bf-40f2-8e25-adc56462f100\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " Jul 08 18:59:06.134059 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.133878 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-workload-certs\") pod \"89588de2-47bf-40f2-8e25-adc56462f100\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " Jul 08 18:59:06.134059 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.133922 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89588de2-47bf-40f2-8e25-adc56462f100-istio-podinfo\") pod \"89588de2-47bf-40f2-8e25-adc56462f100\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " Jul 08 18:59:06.134059 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.133947 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89588de2-47bf-40f2-8e25-adc56462f100-istio-token\") pod \"89588de2-47bf-40f2-8e25-adc56462f100\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " Jul 08 18:59:06.134059 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.133986 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-istio-data\") pod \"89588de2-47bf-40f2-8e25-adc56462f100\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " Jul 08 18:59:06.134059 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.134025 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvqqh\" (UniqueName: \"kubernetes.io/projected/89588de2-47bf-40f2-8e25-adc56462f100-kube-api-access-fvqqh\") pod \"89588de2-47bf-40f2-8e25-adc56462f100\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " Jul 08 18:59:06.134331 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.134067 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-istio-envoy\") pod \"89588de2-47bf-40f2-8e25-adc56462f100\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " Jul 08 18:59:06.134331 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.134108 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-credential-socket\") pod \"89588de2-47bf-40f2-8e25-adc56462f100\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " Jul 08 18:59:06.134331 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.134168 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-workload-socket\") pod \"89588de2-47bf-40f2-8e25-adc56462f100\" (UID: \"89588de2-47bf-40f2-8e25-adc56462f100\") " Jul 08 18:59:06.134331 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.134194 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "89588de2-47bf-40f2-8e25-adc56462f100" (UID: "89588de2-47bf-40f2-8e25-adc56462f100"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:59:06.134331 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.134304 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89588de2-47bf-40f2-8e25-adc56462f100-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "89588de2-47bf-40f2-8e25-adc56462f100" (UID: "89588de2-47bf-40f2-8e25-adc56462f100"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:59:06.134579 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.134444 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "89588de2-47bf-40f2-8e25-adc56462f100" (UID: "89588de2-47bf-40f2-8e25-adc56462f100"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:59:06.134579 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.134516 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "89588de2-47bf-40f2-8e25-adc56462f100" (UID: "89588de2-47bf-40f2-8e25-adc56462f100"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:59:06.134579 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.134565 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-workload-certs\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:59:06.134717 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.134581 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-workload-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:59:06.134717 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.134594 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89588de2-47bf-40f2-8e25-adc56462f100-istiod-ca-cert\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:59:06.135420 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.135394 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-istio-data" (OuterVolumeSpecName: "istio-data") pod "89588de2-47bf-40f2-8e25-adc56462f100" (UID: "89588de2-47bf-40f2-8e25-adc56462f100"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:59:06.136447 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.136416 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/89588de2-47bf-40f2-8e25-adc56462f100-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "89588de2-47bf-40f2-8e25-adc56462f100" (UID: "89588de2-47bf-40f2-8e25-adc56462f100"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:59:06.136554 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.136501 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "89588de2-47bf-40f2-8e25-adc56462f100" (UID: "89588de2-47bf-40f2-8e25-adc56462f100"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:59:06.136720 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.136694 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89588de2-47bf-40f2-8e25-adc56462f100-istio-token" (OuterVolumeSpecName: "istio-token") pod "89588de2-47bf-40f2-8e25-adc56462f100" (UID: "89588de2-47bf-40f2-8e25-adc56462f100"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:59:06.136806 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.136720 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89588de2-47bf-40f2-8e25-adc56462f100-kube-api-access-fvqqh" (OuterVolumeSpecName: "kube-api-access-fvqqh") pod "89588de2-47bf-40f2-8e25-adc56462f100" (UID: "89588de2-47bf-40f2-8e25-adc56462f100"). InnerVolumeSpecName "kube-api-access-fvqqh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:59:06.235451 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.235410 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvqqh\" (UniqueName: \"kubernetes.io/projected/89588de2-47bf-40f2-8e25-adc56462f100-kube-api-access-fvqqh\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:59:06.235451 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.235443 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-istio-envoy\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:59:06.235451 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.235456 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-credential-socket\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:59:06.235682 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.235468 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89588de2-47bf-40f2-8e25-adc56462f100-istio-podinfo\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:59:06.235682 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.235479 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89588de2-47bf-40f2-8e25-adc56462f100-istio-token\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:59:06.235682 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.235490 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89588de2-47bf-40f2-8e25-adc56462f100-istio-data\") on node \"ip-10-0-134-97.ec2.internal\" DevicePath \"\"" Jul 08 18:59:06.615235 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.615202 2574 generic.go:356] "Generic (PLEG): container finished" podID="89588de2-47bf-40f2-8e25-adc56462f100" containerID="5c488b75ebb0f3f31250a01aa960a50e2455b49c3cecb33b026fdbe2975e624a" exitCode=0 Jul 08 18:59:06.615415 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.615256 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" Jul 08 18:59:06.615415 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.615286 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" event={"ID":"89588de2-47bf-40f2-8e25-adc56462f100","Type":"ContainerDied","Data":"5c488b75ebb0f3f31250a01aa960a50e2455b49c3cecb33b026fdbe2975e624a"} Jul 08 18:59:06.615415 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.615340 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt" event={"ID":"89588de2-47bf-40f2-8e25-adc56462f100","Type":"ContainerDied","Data":"c059d58e13abb6bd889a15081adf164473041cae44056d4ebc4d389265abbd64"} Jul 08 18:59:06.615415 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.615369 2574 scope.go:117] "RemoveContainer" containerID="5c488b75ebb0f3f31250a01aa960a50e2455b49c3cecb33b026fdbe2975e624a" Jul 08 18:59:06.624716 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.624694 2574 scope.go:117] "RemoveContainer" containerID="8c65bc206737a74712d2426d12301b5500acd45fe6819696320e08af7842599e" Jul 08 18:59:06.632672 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.632587 2574 scope.go:117] "RemoveContainer" containerID="5c488b75ebb0f3f31250a01aa960a50e2455b49c3cecb33b026fdbe2975e624a" Jul 08 18:59:06.632834 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:59:06.632817 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c488b75ebb0f3f31250a01aa960a50e2455b49c3cecb33b026fdbe2975e624a\": container with ID starting with 5c488b75ebb0f3f31250a01aa960a50e2455b49c3cecb33b026fdbe2975e624a not found: ID does not exist" containerID="5c488b75ebb0f3f31250a01aa960a50e2455b49c3cecb33b026fdbe2975e624a" Jul 08 18:59:06.632927 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.632843 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c488b75ebb0f3f31250a01aa960a50e2455b49c3cecb33b026fdbe2975e624a"} err="failed to get container status \"5c488b75ebb0f3f31250a01aa960a50e2455b49c3cecb33b026fdbe2975e624a\": rpc error: code = NotFound desc = could not find container \"5c488b75ebb0f3f31250a01aa960a50e2455b49c3cecb33b026fdbe2975e624a\": container with ID starting with 5c488b75ebb0f3f31250a01aa960a50e2455b49c3cecb33b026fdbe2975e624a not found: ID does not exist" Jul 08 18:59:06.632976 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.632930 2574 scope.go:117] "RemoveContainer" containerID="8c65bc206737a74712d2426d12301b5500acd45fe6819696320e08af7842599e" Jul 08 18:59:06.633166 ip-10-0-134-97 kubenswrapper[2574]: E0708 18:59:06.633146 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c65bc206737a74712d2426d12301b5500acd45fe6819696320e08af7842599e\": container with ID starting with 8c65bc206737a74712d2426d12301b5500acd45fe6819696320e08af7842599e not found: ID does not exist" containerID="8c65bc206737a74712d2426d12301b5500acd45fe6819696320e08af7842599e" Jul 08 18:59:06.633221 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.633175 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c65bc206737a74712d2426d12301b5500acd45fe6819696320e08af7842599e"} err="failed to get container status \"8c65bc206737a74712d2426d12301b5500acd45fe6819696320e08af7842599e\": rpc error: code = NotFound desc = could not find container \"8c65bc206737a74712d2426d12301b5500acd45fe6819696320e08af7842599e\": container with ID starting with 8c65bc206737a74712d2426d12301b5500acd45fe6819696320e08af7842599e not found: ID does not exist" Jul 08 18:59:06.638413 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.638394 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt"] Jul 08 18:59:06.642161 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:06.642142 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-c926a5-openshift-default-789b989667-hfrtt"] Jul 08 18:59:07.027378 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:07.027294 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-578597fb9c-5pzd7_5494476b-7149-447d-bf7d-c28859c1f24b/copy-manifests/0.log" Jul 08 18:59:07.033496 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:07.033471 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-578597fb9c-5pzd7_5494476b-7149-447d-bf7d-c28859c1f24b/manager/0.log" Jul 08 18:59:07.770389 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:07.770357 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89588de2-47bf-40f2-8e25-adc56462f100" path="/var/lib/kubelet/pods/89588de2-47bf-40f2-8e25-adc56462f100/volumes" Jul 08 18:59:09.934525 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:09.934494 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rdjks_32ca5be3-1d43-41f4-9192-d06ff4660b5d/istio-proxy/0.log" Jul 08 18:59:10.056495 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:10.056470 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-rkbht_cc6bb8b8-82ac-4b33-85ae-435ae7615f1f/discovery/0.log" Jul 08 18:59:10.783548 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:10.783515 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76f778dfc5-nqd5b_8b408e1d-752d-4537-bbfb-145a3520850f/router/0.log" Jul 08 18:59:11.256727 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:11.256697 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-ds24w_e1eb580e-486c-40e6-8ddf-a34ae825e3b6/main/0.log" Jul 08 18:59:11.374256 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:11.374231 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-s7hhw_43b85d6e-5201-4d92-bba3-b4df7346b1e2/main/0.log" Jul 08 18:59:14.708406 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:14.708366 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:59:15.849598 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:15.849566 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:59:15.916889 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:15.916844 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" limit=1024 Jul 08 18:59:18.824698 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:18.824670 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xw9f2_506c4790-d442-4464-8b83-06d2891a6df0/global-pull-secret-syncer/0.log" Jul 08 18:59:18.867672 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:18.867651 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8svs8_a4e7ee8d-935d-4f12-a691-c123e9a5fd27/konnectivity-agent/0.log" Jul 08 18:59:18.977104 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:18.977076 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-97.ec2.internal_069618fd9935d30ef59a0acde000b9cb/haproxy/0.log" Jul 08 18:59:26.290099 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:26.290069 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-8bsph_b76b8c21-85aa-43de-b1f7-4224e372467a/cluster-monitoring-operator/0.log" Jul 08 18:59:26.329586 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:26.329553 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-lrbbc_3f64fdac-6907-4dba-bc6d-491cda836b65/kube-state-metrics/0.log" Jul 08 18:59:26.354528 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:26.354503 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-lrbbc_3f64fdac-6907-4dba-bc6d-491cda836b65/kube-rbac-proxy-main/0.log" Jul 08 18:59:26.376486 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:26.376458 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-lrbbc_3f64fdac-6907-4dba-bc6d-491cda836b65/kube-rbac-proxy-self/0.log" Jul 08 18:59:26.548755 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:26.548674 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b2r5c_a63305b0-9222-492b-8f3d-b3860f225b31/node-exporter/0.log" Jul 08 18:59:26.573130 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:26.573088 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b2r5c_a63305b0-9222-492b-8f3d-b3860f225b31/kube-rbac-proxy/0.log" Jul 08 18:59:26.596498 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:26.596475 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b2r5c_a63305b0-9222-492b-8f3d-b3860f225b31/init-textfile/0.log" Jul 08 18:59:26.791444 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:26.791420 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2f2c60e0-cbc3-48b8-bc89-74c63d74717a/prometheus/0.log" Jul 08 18:59:26.814407 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:26.814386 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2f2c60e0-cbc3-48b8-bc89-74c63d74717a/config-reloader/0.log" Jul 08 18:59:26.834888 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:26.834863 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2f2c60e0-cbc3-48b8-bc89-74c63d74717a/thanos-sidecar/0.log" Jul 08 18:59:26.855738 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:26.855718 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2f2c60e0-cbc3-48b8-bc89-74c63d74717a/kube-rbac-proxy-web/0.log" Jul 08 18:59:26.877686 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:26.877669 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2f2c60e0-cbc3-48b8-bc89-74c63d74717a/kube-rbac-proxy/0.log" Jul 08 18:59:26.899364 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:26.899343 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2f2c60e0-cbc3-48b8-bc89-74c63d74717a/kube-rbac-proxy-thanos/0.log" Jul 08 18:59:26.920497 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:26.920475 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2f2c60e0-cbc3-48b8-bc89-74c63d74717a/init-config-reloader/0.log" Jul 08 18:59:27.042324 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:27.042300 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-859bf86486-s6rv4_1f231c29-f57a-48b2-b71e-c5c2f80be098/telemeter-client/0.log" Jul 08 18:59:27.064352 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:27.064328 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-859bf86486-s6rv4_1f231c29-f57a-48b2-b71e-c5c2f80be098/reload/0.log" Jul 08 18:59:27.087431 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:27.087351 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-859bf86486-s6rv4_1f231c29-f57a-48b2-b71e-c5c2f80be098/kube-rbac-proxy/0.log" Jul 08 18:59:27.125751 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:27.125714 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d868f677b-hlglt_990d6988-42df-4af3-bec7-4b652a019f99/thanos-query/0.log" Jul 08 18:59:27.146062 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:27.146039 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d868f677b-hlglt_990d6988-42df-4af3-bec7-4b652a019f99/kube-rbac-proxy-web/0.log" Jul 08 18:59:27.173112 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:27.173075 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d868f677b-hlglt_990d6988-42df-4af3-bec7-4b652a019f99/kube-rbac-proxy/0.log" Jul 08 18:59:27.194707 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:27.194681 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d868f677b-hlglt_990d6988-42df-4af3-bec7-4b652a019f99/prom-label-proxy/0.log" Jul 08 18:59:27.216068 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:27.216049 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d868f677b-hlglt_990d6988-42df-4af3-bec7-4b652a019f99/kube-rbac-proxy-rules/0.log" Jul 08 18:59:27.236075 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:27.236055 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d868f677b-hlglt_990d6988-42df-4af3-bec7-4b652a019f99/kube-rbac-proxy-metrics/0.log" Jul 08 18:59:28.718677 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.718645 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4"] Jul 08 18:59:28.719076 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.718991 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89588de2-47bf-40f2-8e25-adc56462f100" containerName="istio-proxy" Jul 08 18:59:28.719076 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.719002 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="89588de2-47bf-40f2-8e25-adc56462f100" containerName="istio-proxy" Jul 08 18:59:28.719076 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.719011 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a728e58d-4a3c-4dfc-a229-e6c6ed9654db" containerName="istio-proxy" Jul 08 18:59:28.719076 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.719017 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a728e58d-4a3c-4dfc-a229-e6c6ed9654db" containerName="istio-proxy" Jul 08 18:59:28.719076 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.719022 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0764d2ee-ec8d-4a91-8f67-b8242388b672" containerName="istio-proxy" Jul 08 18:59:28.719076 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.719027 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0764d2ee-ec8d-4a91-8f67-b8242388b672" containerName="istio-proxy" Jul 08 18:59:28.719076 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.719032 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0764d2ee-ec8d-4a91-8f67-b8242388b672" containerName="istio-proxy" Jul 08 18:59:28.719076 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.719037 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0764d2ee-ec8d-4a91-8f67-b8242388b672" containerName="istio-proxy" Jul 08 18:59:28.719076 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.719056 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89588de2-47bf-40f2-8e25-adc56462f100" containerName="istio-proxy" Jul 08 18:59:28.719076 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.719061 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="89588de2-47bf-40f2-8e25-adc56462f100" containerName="istio-proxy" Jul 08 18:59:28.719402 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.719121 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="89588de2-47bf-40f2-8e25-adc56462f100" containerName="istio-proxy" Jul 08 18:59:28.719402 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.719129 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0764d2ee-ec8d-4a91-8f67-b8242388b672" containerName="istio-proxy" Jul 08 18:59:28.719402 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.719135 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="89588de2-47bf-40f2-8e25-adc56462f100" containerName="istio-proxy" Jul 08 18:59:28.719402 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.719141 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0764d2ee-ec8d-4a91-8f67-b8242388b672" containerName="istio-proxy" Jul 08 18:59:28.723593 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.723574 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:28.728295 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.728274 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6qtft\"/\"openshift-service-ca.crt\"" Jul 08 18:59:28.728396 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.728326 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6qtft\"/\"kube-root-ca.crt\"" Jul 08 18:59:28.729179 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.729162 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6qtft\"/\"default-dockercfg-4257n\"" Jul 08 18:59:28.738656 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.738635 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4"] Jul 08 18:59:28.845386 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.845356 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrksz\" (UniqueName: \"kubernetes.io/projected/b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432-kube-api-access-hrksz\") pod \"perf-node-gather-daemonset-qpqv4\" (UID: \"b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:28.845581 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.845416 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432-sys\") pod \"perf-node-gather-daemonset-qpqv4\" (UID: \"b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:28.845581 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.845441 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432-lib-modules\") pod \"perf-node-gather-daemonset-qpqv4\" (UID: \"b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:28.845581 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.845495 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432-proc\") pod \"perf-node-gather-daemonset-qpqv4\" (UID: \"b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:28.845733 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.845613 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432-podres\") pod \"perf-node-gather-daemonset-qpqv4\" (UID: \"b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:28.946396 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.946365 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432-proc\") pod \"perf-node-gather-daemonset-qpqv4\" (UID: \"b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:28.946568 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.946450 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432-podres\") pod \"perf-node-gather-daemonset-qpqv4\" (UID: \"b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:28.946568 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.946484 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432-proc\") pod \"perf-node-gather-daemonset-qpqv4\" (UID: \"b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:28.946568 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.946494 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrksz\" (UniqueName: \"kubernetes.io/projected/b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432-kube-api-access-hrksz\") pod \"perf-node-gather-daemonset-qpqv4\" (UID: \"b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:28.946735 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.946577 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432-sys\") pod \"perf-node-gather-daemonset-qpqv4\" (UID: \"b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:28.946735 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.946600 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432-podres\") pod \"perf-node-gather-daemonset-qpqv4\" (UID: \"b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:28.946735 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.946612 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432-lib-modules\") pod \"perf-node-gather-daemonset-qpqv4\" (UID: \"b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:28.946735 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.946647 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432-sys\") pod \"perf-node-gather-daemonset-qpqv4\" (UID: \"b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:28.946937 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.946798 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432-lib-modules\") pod \"perf-node-gather-daemonset-qpqv4\" (UID: \"b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:28.955479 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:28.955455 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrksz\" (UniqueName: \"kubernetes.io/projected/b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432-kube-api-access-hrksz\") pod \"perf-node-gather-daemonset-qpqv4\" (UID: \"b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:29.033832 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:29.033752 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:29.161677 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:29.161646 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4"] Jul 08 18:59:29.164841 ip-10-0-134-97 kubenswrapper[2574]: W0708 18:59:29.164807 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb1ee6ecd_0afe_4fb6_9cf5_cfcd15290432.slice/crio-b52733b118664e6ff30068b55d97e283df403e3aef699a7e0d48d811166b815d WatchSource:0}: Error finding container b52733b118664e6ff30068b55d97e283df403e3aef699a7e0d48d811166b815d: Status 404 returned error can't find the container with id b52733b118664e6ff30068b55d97e283df403e3aef699a7e0d48d811166b815d Jul 08 18:59:29.721628 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:29.721591 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" event={"ID":"b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432","Type":"ContainerStarted","Data":"03be698edb69020c3cff6d34ab4ab08594324762e81add92e9cc35786fd75e64"} Jul 08 18:59:29.722069 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:29.721641 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" event={"ID":"b1ee6ecd-0afe-4fb6-9cf5-cfcd15290432","Type":"ContainerStarted","Data":"b52733b118664e6ff30068b55d97e283df403e3aef699a7e0d48d811166b815d"} Jul 08 18:59:29.722069 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:29.721681 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:29.736868 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:29.736678 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" podStartSLOduration=1.736662154 podStartE2EDuration="1.736662154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:59:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:59:29.736280273 +0000 UTC m=+2490.487589748" watchObservedRunningTime="2026-07-08 18:59:29.736662154 +0000 UTC m=+2490.487971641" Jul 08 18:59:31.295655 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:31.295629 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cz6kw_1e9502fd-4d0e-4186-84b2-76b6db2593b1/dns/0.log" Jul 08 18:59:31.331483 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:31.331457 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cz6kw_1e9502fd-4d0e-4186-84b2-76b6db2593b1/kube-rbac-proxy/0.log" Jul 08 18:59:31.451087 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:31.451060 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rxdnb_dfd1b4d7-5544-40c0-af2a-f22db9888c91/dns-node-resolver/0.log" Jul 08 18:59:31.948760 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:31.948730 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-56896c69c6-2fsq8_e46fbbf2-24a7-447c-8904-7e0d7f566bdd/registry/0.log" Jul 08 18:59:31.989752 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:31.989722 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-smjlq_ec2217c0-db7c-4660-a6f3-990b90cbf03d/node-ca/0.log" Jul 08 18:59:32.944083 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:32.944053 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rdjks_32ca5be3-1d43-41f4-9192-d06ff4660b5d/istio-proxy/0.log" Jul 08 18:59:33.038477 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:33.038446 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-rkbht_cc6bb8b8-82ac-4b33-85ae-435ae7615f1f/discovery/0.log" Jul 08 18:59:33.276531 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:33.276447 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76f778dfc5-nqd5b_8b408e1d-752d-4537-bbfb-145a3520850f/router/0.log" Jul 08 18:59:33.959965 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:33.959936 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9fl2r_58fbd0df-e933-4bee-9916-4b786b0205a8/serve-healthcheck-canary/0.log" Jul 08 18:59:34.435318 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:34.435286 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-qkmvz_07b57a1f-a5d2-4bfa-81ac-188e4c33816d/insights-operator/0.log" Jul 08 18:59:34.436487 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:34.436468 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-qkmvz_07b57a1f-a5d2-4bfa-81ac-188e4c33816d/insights-operator/1.log" Jul 08 18:59:34.598585 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:34.598555 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xdvmc_67415b1e-f842-44cf-89dc-ce26a26d56db/kube-rbac-proxy/0.log" Jul 08 18:59:34.619459 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:34.619434 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xdvmc_67415b1e-f842-44cf-89dc-ce26a26d56db/exporter/0.log" Jul 08 18:59:34.644576 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:34.644552 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xdvmc_67415b1e-f842-44cf-89dc-ce26a26d56db/extractor/0.log" Jul 08 18:59:35.735798 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:35.735763 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-qpqv4" Jul 08 18:59:37.170411 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:37.170382 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-578597fb9c-5pzd7_5494476b-7149-447d-bf7d-c28859c1f24b/manager/0.log" Jul 08 18:59:37.194185 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:37.194162 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-578597fb9c-5pzd7_5494476b-7149-447d-bf7d-c28859c1f24b/copy-manifests/0.log" Jul 08 18:59:38.849663 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:38.849634 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-56665c6f48-jc6wv_4925964a-8302-4165-af39-acb1fe708708/manager/0.log" Jul 08 18:59:39.226802 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:39.226772 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rdjks_32ca5be3-1d43-41f4-9192-d06ff4660b5d/istio-proxy/0.log" Jul 08 18:59:39.317003 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:39.316977 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-rkbht_cc6bb8b8-82ac-4b33-85ae-435ae7615f1f/discovery/0.log" Jul 08 18:59:39.499947 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:39.499865 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76f778dfc5-nqd5b_8b408e1d-752d-4537-bbfb-145a3520850f/router/0.log" Jul 08 18:59:40.617803 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:40.617773 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rdjks_32ca5be3-1d43-41f4-9192-d06ff4660b5d/istio-proxy/0.log" Jul 08 18:59:40.709419 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:40.709379 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-rkbht_cc6bb8b8-82ac-4b33-85ae-435ae7615f1f/discovery/0.log" Jul 08 18:59:40.918880 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:40.918782 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76f778dfc5-nqd5b_8b408e1d-752d-4537-bbfb-145a3520850f/router/0.log" Jul 08 18:59:45.711861 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:45.711817 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tmdn8_33915cb2-f6ca-4540-9425-ff653a28c8a2/kube-multus-additional-cni-plugins/0.log" Jul 08 18:59:45.732944 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:45.732924 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tmdn8_33915cb2-f6ca-4540-9425-ff653a28c8a2/egress-router-binary-copy/0.log" Jul 08 18:59:45.759633 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:45.759609 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tmdn8_33915cb2-f6ca-4540-9425-ff653a28c8a2/cni-plugins/0.log" Jul 08 18:59:45.783504 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:45.783483 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tmdn8_33915cb2-f6ca-4540-9425-ff653a28c8a2/bond-cni-plugin/0.log" Jul 08 18:59:45.806100 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:45.806074 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tmdn8_33915cb2-f6ca-4540-9425-ff653a28c8a2/routeoverride-cni/0.log" Jul 08 18:59:45.827507 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:45.827473 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tmdn8_33915cb2-f6ca-4540-9425-ff653a28c8a2/whereabouts-cni-bincopy/0.log" Jul 08 18:59:45.848770 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:45.848746 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tmdn8_33915cb2-f6ca-4540-9425-ff653a28c8a2/whereabouts-cni/0.log" Jul 08 18:59:45.880613 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:45.880585 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dkrkt_ff6bcc41-a870-4794-96f3-418b31e9a666/kube-multus/0.log" Jul 08 18:59:45.992807 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:45.992740 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ts8sd_0d868453-b10b-4dd9-b843-91de3f1b48d3/network-metrics-daemon/0.log" Jul 08 18:59:46.015449 ip-10-0-134-97 kubenswrapper[2574]: I0708 18:59:46.015421 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ts8sd_0d868453-b10b-4dd9-b843-91de3f1b48d3/kube-rbac-proxy/0.log"