Jul 02 19:05:51.735768 ip-10-0-133-161 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 02 19:05:51.735780 ip-10-0-133-161 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 02 19:05:51.735787 ip-10-0-133-161 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 02 19:05:51.736030 ip-10-0-133-161 systemd[1]: Failed to start Kubernetes Kubelet. Jul 02 19:06:01.819394 ip-10-0-133-161 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 02 19:06:01.819408 ip-10-0-133-161 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 8468f0cb474e4bfca8cf2be7caaca7aa -- Jul 02 19:07:37.068672 ip-10-0-133-161 systemd[1]: Starting Kubernetes Kubelet... Jul 02 19:07:37.605934 ip-10-0-133-161 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 19:07:37.605934 ip-10-0-133-161 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 19:07:37.605934 ip-10-0-133-161 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 19:07:37.605934 ip-10-0-133-161 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 19:07:37.605934 ip-10-0-133-161 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 19:07:37.605934 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.519221 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 19:07:37.605934 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525925 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:07:37.605934 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525936 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:07:37.605934 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525940 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:07:37.605934 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525943 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:07:37.605934 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525946 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:07:37.605934 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525949 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:07:37.605934 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525952 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:07:37.605934 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525955 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525958 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525966 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525969 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525972 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525976 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525980 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525982 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525985 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525988 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525991 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525994 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.525997 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526000 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526003 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526006 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526010 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526014 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526017 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526020 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:07:37.617119 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526023 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:07:37.599948 ip-10-0-133-161 systemd[1]: Started Kubernetes Kubelet. Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526027 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526030 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526033 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526035 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526038 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526041 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526044 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526047 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526050 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526053 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526056 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526058 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526061 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526065 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526068 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526070 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526073 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526078 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:07:37.620496 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526081 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526084 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526087 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526090 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526093 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526096 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526099 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526102 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526105 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526108 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526111 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526113 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526116 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526119 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526122 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526125 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526130 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526133 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526136 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:07:37.623359 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526139 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526141 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526144 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526146 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526149 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526152 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526154 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526157 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526159 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526162 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526165 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526167 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526170 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526172 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526175 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526177 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526180 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526182 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526185 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526187 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:07:37.625638 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526190 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526192 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526195 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526197 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526200 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526202 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526205 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526208 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526211 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526213 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526215 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526218 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526220 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526223 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526915 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526922 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526925 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526928 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526930 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526933 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:07:37.627994 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526936 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526939 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526941 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526945 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526947 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526950 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526953 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526955 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526958 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526962 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526966 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526969 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526972 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526974 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526977 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526979 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526982 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526985 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526987 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:07:37.629885 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526990 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526993 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526996 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.526998 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527002 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527004 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527007 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527010 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527013 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527015 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527018 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527020 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527023 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527026 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527028 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527031 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527033 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527037 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527040 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527042 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:07:37.632326 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527045 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527048 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527050 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527052 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527056 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527058 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527061 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527063 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527066 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527070 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527074 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527077 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527080 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527085 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527088 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527091 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527093 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527096 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527099 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:07:37.648519 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527101 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527104 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527107 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527110 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527113 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527115 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527118 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527121 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527123 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527126 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527128 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527131 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527134 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527138 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527141 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527143 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527146 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527149 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527151 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527153 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:07:37.650793 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527156 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527158 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527161 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527163 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527166 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527169 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527174 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527178 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527181 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527183 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527186 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527189 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527192 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527194 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527197 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527553 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527559 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527561 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527564 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:07:37.653474 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527567 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527570 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527572 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527575 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527579 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527581 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527584 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527586 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527589 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527591 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527594 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527597 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527599 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527602 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527605 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527607 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527610 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527612 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527615 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527621 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:07:37.692431 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527623 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527626 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527628 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527631 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527634 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527636 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527639 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527642 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527644 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527647 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527649 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527652 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527654 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527656 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527659 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527661 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527664 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527667 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527670 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:07:37.816811 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527673 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527675 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527677 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527680 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527682 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527685 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527687 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527690 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527692 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527697 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527700 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527704 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527708 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527711 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527713 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527716 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527719 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527721 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527725 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:07:37.817955 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527728 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527731 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527733 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527736 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527740 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527742 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527745 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527748 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527750 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527752 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527755 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527757 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527760 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527763 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527766 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527768 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527771 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527775 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527778 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527781 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:07:37.818791 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527785 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527788 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527792 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527796 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527800 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527805 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527823 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527826 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527829 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527831 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527834 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527836 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527839 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527841 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527844 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527847 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.527853 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527918 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527925 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527932 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527936 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 19:07:37.819803 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527940 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527943 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527947 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527951 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527955 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527958 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527961 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527964 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527967 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527970 2578 flags.go:64] FLAG: --cgroup-root="" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527973 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527976 2578 flags.go:64] FLAG: --client-ca-file="" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527978 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527981 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527985 2578 flags.go:64] FLAG: --cluster-domain="" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527988 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527993 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527996 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.527999 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528003 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528005 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528008 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528012 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528014 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 02 19:07:37.821016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528017 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528021 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528024 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528027 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528031 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528034 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528037 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528040 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528043 2578 flags.go:64] FLAG: --enable-server="true" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528046 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528790 2578 flags.go:64] FLAG: --event-burst="100" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528795 2578 flags.go:64] FLAG: --event-qps="50" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528798 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528808 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528811 2578 flags.go:64] FLAG: --eviction-hard="" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528819 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528822 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528825 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528829 2578 flags.go:64] FLAG: --eviction-soft="" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528832 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528834 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528837 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528840 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528843 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528847 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 19:07:37.821935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528850 2578 flags.go:64] FLAG: --feature-gates="" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528853 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528856 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528859 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528862 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528866 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528869 2578 flags.go:64] FLAG: --help="false" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528872 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-133-161.ec2.internal" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528875 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528878 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528882 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528885 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528888 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528891 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528894 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528896 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528899 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528902 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528905 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528908 2578 flags.go:64] FLAG: --kube-reserved="" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528911 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528914 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528917 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528920 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 19:07:37.822783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528923 2578 flags.go:64] FLAG: --lock-file="" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528925 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528928 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528931 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528936 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528939 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528941 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528945 2578 flags.go:64] FLAG: --logging-format="text" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528948 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528951 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528954 2578 flags.go:64] FLAG: --manifest-url="" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528957 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528961 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528964 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528968 2578 flags.go:64] FLAG: --max-pods="110" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528971 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528974 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528977 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528980 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528984 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528987 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528989 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528996 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.528999 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529002 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 19:07:37.823496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529005 2578 flags.go:64] FLAG: --pod-cidr="" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529007 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529013 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529015 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529018 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529022 2578 flags.go:64] FLAG: --port="10250" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529025 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529027 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-023045eaeb9a32b94" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529031 2578 flags.go:64] FLAG: --qos-reserved="" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529033 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529036 2578 flags.go:64] FLAG: --register-node="true" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529039 2578 flags.go:64] FLAG: --register-with-taints="" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529042 2578 flags.go:64] FLAG: --registry-burst="10" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529045 2578 flags.go:64] FLAG: --registry-qps="5" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529048 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529052 2578 flags.go:64] FLAG: --reserved-memory="" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529055 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529059 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529061 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529064 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529067 2578 flags.go:64] FLAG: --runonce="false" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529070 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529073 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529076 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529079 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 19:07:37.824509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529084 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529087 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529090 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529093 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529096 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529099 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529101 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529104 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529107 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529110 2578 flags.go:64] FLAG: --system-cgroups="" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529112 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529117 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529120 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529123 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529127 2578 flags.go:64] FLAG: --tls-min-version="" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529130 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529133 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529135 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529138 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529141 2578 flags.go:64] FLAG: --v="2" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529144 2578 flags.go:64] FLAG: --version="false" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529148 2578 flags.go:64] FLAG: --vmodule="" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529153 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.529156 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529262 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:07:37.825527 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529266 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529269 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529272 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529275 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529277 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529280 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529283 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529286 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529289 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529291 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529294 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529297 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529299 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529301 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529304 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529306 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529309 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529312 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529314 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:07:37.968015 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529316 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529319 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529322 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529324 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529327 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529329 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529332 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529334 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529337 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529339 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529343 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529345 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529348 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529350 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529353 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529355 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529358 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529360 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529363 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529365 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:07:37.968807 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529369 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529371 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529374 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529376 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529379 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529381 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529384 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529386 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529389 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529391 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529408 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529410 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529413 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529415 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529418 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529421 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529424 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529426 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529429 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529431 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:07:37.969614 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529434 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529436 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529441 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529443 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529446 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529449 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529451 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529454 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529456 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529459 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529461 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529464 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529468 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529470 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529473 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529476 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529478 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529481 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529483 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529487 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:07:37.970219 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529491 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529494 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529497 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529501 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529505 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529508 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529511 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529514 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529518 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529520 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529523 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529525 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529528 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529531 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529536 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529538 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529541 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529543 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:07:37.970801 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.529546 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:07:37.971325 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.530310 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 19:07:37.971325 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.538027 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 19:07:37.971325 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.538042 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 19:07:37.971325 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.538921 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 19:07:37.971325 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.538932 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 19:07:37.971325 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539008 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:07:37.971325 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539014 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:07:37.971325 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539019 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:07:37.971325 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539024 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:07:37.971325 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539028 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:07:37.971325 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539033 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:07:37.971325 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539037 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:07:37.971325 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539041 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:07:37.971325 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539046 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:07:37.971325 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539049 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539053 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539057 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539061 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539065 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539070 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539074 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539078 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539082 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539086 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539090 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539094 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539098 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539102 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539106 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539109 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539113 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539117 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539121 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:07:37.971767 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539125 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539129 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539133 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539136 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539140 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539144 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539148 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539152 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539155 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539159 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539164 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539168 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539172 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539176 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539180 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539184 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539188 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539191 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539196 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539205 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:07:37.972317 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539209 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539213 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539217 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539221 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539227 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539234 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539238 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539242 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539247 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539250 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539254 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539259 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539264 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539269 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539274 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539278 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539282 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539287 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539293 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:07:37.972861 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539297 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539301 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539305 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539309 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539313 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539316 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539320 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539324 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539328 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539332 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539335 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539339 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539345 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539348 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539352 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539356 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539360 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539364 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539369 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539373 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:07:37.973452 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539377 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:07:37.973987 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539381 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:07:37.973987 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539384 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:07:37.973987 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539388 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:07:37.973987 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539406 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:07:37.973987 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539411 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:07:37.973987 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539415 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:07:37.973987 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539419 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:07:37.973987 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539423 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:07:37.973987 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539427 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:07:37.973987 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539431 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:07:37.973987 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539434 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:07:37.973987 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.539441 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 19:07:37.973987 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539599 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:07:37.973987 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539606 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:07:37.973987 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539611 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539615 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539619 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539623 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539627 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539631 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539636 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539639 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539643 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539648 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539654 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539658 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539662 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539667 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539670 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539680 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539683 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539688 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539691 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539695 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:07:37.974462 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539699 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539703 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539707 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539711 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539716 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539720 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539724 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539727 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539730 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539733 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539736 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539738 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539741 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539744 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539747 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539750 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539753 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539756 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539758 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:07:37.974982 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539761 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539764 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539766 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539770 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539774 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539777 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539780 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539783 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539787 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539789 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539792 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539796 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539800 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539804 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539808 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539812 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539816 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539820 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539823 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539827 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:07:37.975578 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539831 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539835 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539839 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539842 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539846 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539850 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539854 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539858 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539862 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539866 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539869 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539873 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539877 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539881 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539884 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539890 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539893 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539899 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539903 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539909 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:07:37.976117 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539915 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:07:37.976700 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539919 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:07:37.976700 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539923 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:07:37.976700 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539927 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:07:37.976700 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539930 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:07:37.976700 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539934 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:07:37.976700 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539938 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:07:37.976700 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539941 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:07:37.976700 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539945 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:07:37.976700 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539949 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:07:37.976700 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539952 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:07:37.976700 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539956 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:07:37.976700 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539959 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:07:37.976700 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539963 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:07:37.976700 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539966 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:07:37.976700 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539970 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:07:37.976700 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539974 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:07:37.976700 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:37.539977 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:07:37.977186 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.539983 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 19:07:37.977186 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.540104 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 19:07:37.977186 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.544343 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 19:07:37.977186 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.546075 2578 server.go:1019] "Starting client certificate rotation" Jul 02 19:07:37.977186 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.546168 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 19:07:37.977186 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.546208 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 19:07:37.977186 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.550326 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 19:07:37.977186 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.551341 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 19:07:37.977186 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.562348 2578 log.go:25] "Validated CRI v1 runtime API" Jul 02 19:07:37.977186 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.564681 2578 log.go:25] "Validated CRI v1 image API" Jul 02 19:07:37.977186 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.565481 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 19:07:37.977186 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.568040 2578 fs.go:135] Filesystem UUIDs: map[7409e5b9-4bb0-47ee-8055-ba046555a709:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8b7a178a-8159-48ac-a7b4-ded94a620e16:/dev/nvme0n1p3] Jul 02 19:07:37.977570 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.568053 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 19:07:37.977570 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.575212 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.575547 2578 manager.go:216] Machine: {Timestamp:2026-07-02 19:07:37.574241163 +0000 UTC m=+0.377056626 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097171 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28f111360661442e689d8d58cdaac1 SystemUUID:ec28f111-3606-6144-2e68-9d8d58cdaac1 BootID:8468f0cb-474e-4bfc-a8cf-2be7caaca7aa Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:91:c4:c0:7e:81 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:91:c4:c0:7e:81 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:18:11:11:4a:57 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.575644 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.575715 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.579152 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.579176 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-161.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.580015 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.580028 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.580035 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.580953 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.582842 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.587484 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.587591 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.587617 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.587626 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.587639 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.587648 2578 kubelet.go:398] "Adding apiserver pod source" Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.587656 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 19:07:37.977728 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.587995 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.588017 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.590491 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.593118 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.593163 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.594721 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.594736 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.594742 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.594748 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.594757 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.594762 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.594770 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.594778 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.594798 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.594805 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.594817 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.594823 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.595741 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.595750 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.598604 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-161.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:37.598674 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:37.598729 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-161.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.599000 2578 server.go:1268] "Started kubelet" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.599063 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.599073 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.599136 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.600260 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.601095 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:37.606490 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.609695 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.609698 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:37.609870 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-161.ec2.internal\" not found" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.610499 2578 factory.go:55] Registering systemd factory Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.610506 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.610508 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.610622 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.610517 2578 factory.go:223] Registration of the systemd container factory successfully Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.610699 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.610709 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.610846 2578 factory.go:153] Registering CRI-O factory Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.610857 2578 factory.go:223] Registration of the crio container factory successfully Jul 02 19:07:37.980231 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:37.609680 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-161.ec2.internal.18be8e9164de1224 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-161.ec2.internal,UID:ip-10-0-133-161.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-161.ec2.internal,},FirstTimestamp:2026-07-02 19:07:37.598972452 +0000 UTC m=+0.401787893,LastTimestamp:2026-07-02 19:07:37.598972452 +0000 UTC m=+0.401787893,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-161.ec2.internal,}" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.610916 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.610934 2578 factory.go:103] Registering Raw factory Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.610947 2578 manager.go:1196] Started watching for new ooms in manager Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.611284 2578 manager.go:349] Starting recovery of all containers Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:37.620750 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:37.620842 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-161.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.622496 2578 manager.go:354] Recovery completed Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.622565 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dmbc5" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.627584 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.629349 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.629373 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.629383 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.629626 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dmbc5" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.629929 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.629937 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.629938 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.629970 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.630636 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.630665 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.630678 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:37.630935 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-161.ec2.internal.18be8e9166adc127 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-161.ec2.internal,UID:ip-10-0-133-161.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-161.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-161.ec2.internal,},FirstTimestamp:2026-07-02 19:07:37.629360423 +0000 UTC m=+0.432175864,LastTimestamp:2026-07-02 19:07:37.629360423 +0000 UTC m=+0.432175864,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-161.ec2.internal,}" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.632792 2578 policy_none.go:49] "None policy: Start" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.632807 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.632820 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.634893 2578 policy_none.go:47] "Start" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.688909 2578 manager.go:337] "Starting Device Plugin manager" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:37.688936 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.688948 2578 server.go:87] "Starting device plugin registration server" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.689162 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.689172 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.689268 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.689349 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.689358 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:37.689750 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:37.689805 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-161.ec2.internal\" not found" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.719478 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.720582 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.720620 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 19:07:37.981682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.720641 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:37.720678 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.724358 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.789421 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.790239 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.790275 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.790286 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.790307 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.801972 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:37.801990 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-161.ec2.internal\": node \"ip-10-0-133-161.ec2.internal\" not found" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.820880 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-161.ec2.internal"] Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.820952 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.822267 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.822292 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.822302 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.824544 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.824659 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.824683 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.825174 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.825198 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.825206 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.825230 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.825245 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.825210 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.827520 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.827540 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.828118 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.828139 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:37.828148 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:37.830659 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-161.ec2.internal\" not found" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:37.870020 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-161.ec2.internal\" not found" node="ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:37.874418 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-161.ec2.internal\" not found" node="ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:37.931607 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-161.ec2.internal\" not found" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.011935 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/05f91772ec7f4ad16d2e21378d51cdee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal\" (UID: \"05f91772ec7f4ad16d2e21378d51cdee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.146694 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.011969 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05f91772ec7f4ad16d2e21378d51cdee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal\" (UID: \"05f91772ec7f4ad16d2e21378d51cdee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.147664 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.011993 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3d1df1679904c800dfb191bda1f933a-config\") pod \"kube-apiserver-proxy-ip-10-0-133-161.ec2.internal\" (UID: \"b3d1df1679904c800dfb191bda1f933a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.147664 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:38.032185 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-161.ec2.internal\" not found" Jul 02 19:07:38.147664 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.114663 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/05f91772ec7f4ad16d2e21378d51cdee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal\" (UID: \"05f91772ec7f4ad16d2e21378d51cdee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.147664 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.114695 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05f91772ec7f4ad16d2e21378d51cdee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal\" (UID: \"05f91772ec7f4ad16d2e21378d51cdee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.147664 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.114714 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3d1df1679904c800dfb191bda1f933a-config\") pod \"kube-apiserver-proxy-ip-10-0-133-161.ec2.internal\" (UID: \"b3d1df1679904c800dfb191bda1f933a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.147664 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.114748 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05f91772ec7f4ad16d2e21378d51cdee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal\" (UID: \"05f91772ec7f4ad16d2e21378d51cdee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.147664 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.114772 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3d1df1679904c800dfb191bda1f933a-config\") pod \"kube-apiserver-proxy-ip-10-0-133-161.ec2.internal\" (UID: \"b3d1df1679904c800dfb191bda1f933a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.147664 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.114753 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/05f91772ec7f4ad16d2e21378d51cdee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal\" (UID: \"05f91772ec7f4ad16d2e21378d51cdee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.147664 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:38.132777 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-161.ec2.internal\" not found" Jul 02 19:07:38.269939 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.177938 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.269939 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.178999 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.269939 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:38.233594 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-161.ec2.internal\" not found" Jul 02 19:07:38.384848 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:38.334035 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-161.ec2.internal\" not found" Jul 02 19:07:38.526458 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:38.435149 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-161.ec2.internal\" not found" Jul 02 19:07:38.526458 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.521299 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.545510 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.545622 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.588583 2578 apiserver.go:52] "Watching apiserver" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.600639 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.601926 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc","openshift-image-registry/node-ca-57hcx","openshift-multus/multus-vtngm","openshift-network-diagnostics/network-check-target-9qvrm","openshift-network-operator/iptables-alerter-ncfg2","openshift-ovn-kubernetes/ovnkube-node-7h54r","kube-system/global-pull-secret-syncer-p2dbb","openshift-cluster-node-tuning-operator/tuned-l5h6q","openshift-dns/node-resolver-jkjv7","openshift-multus/multus-additional-cni-plugins-prfg6","openshift-multus/network-metrics-daemon-rjlz4","kube-system/konnectivity-agent-vqm46"] Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.607082 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.607983 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-57hcx" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.609719 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.609824 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.611382 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.611593 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.611815 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q25kw\"" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.612059 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.612194 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.612540 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.612726 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-69wfj\"" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.612841 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.614910 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-g7gsx\"" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.615068 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.615178 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.615437 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.615626 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.616003 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:38.616045 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.616231 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ncfg2" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.616545 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.617585 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-sys-fs\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.617627 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwshw\" (UniqueName: \"kubernetes.io/projected/c0aec0a3-2014-4d88-a168-33b4b7048bb0-kube-api-access-wwshw\") pod \"node-ca-57hcx\" (UID: \"c0aec0a3-2014-4d88-a168-33b4b7048bb0\") " pod="openshift-image-registry/node-ca-57hcx" Jul 02 19:07:38.628185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.617652 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-var-lib-kubelet\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.629535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.617694 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-multus-conf-dir\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.629535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.617736 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-etc-kubernetes\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.629535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.617783 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5562\" (UniqueName: \"kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562\") pod \"network-check-target-9qvrm\" (UID: \"f43fc6d3-00d7-4153-aea4-423b1d40e67b\") " pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:38.629535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.617812 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.629535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.617833 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-socket-dir\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.629535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.617902 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-device-dir\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.629535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.617930 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c0aec0a3-2014-4d88-a168-33b4b7048bb0-serviceca\") pod \"node-ca-57hcx\" (UID: \"c0aec0a3-2014-4d88-a168-33b4b7048bb0\") " pod="openshift-image-registry/node-ca-57hcx" Jul 02 19:07:38.629535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.617971 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-multus-cni-dir\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.629535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.617996 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fe5b83d0-db4b-4c20-9083-f693a6885494-multus-daemon-config\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.629535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618023 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.629535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618068 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c0aec0a3-2014-4d88-a168-33b4b7048bb0-host\") pod \"node-ca-57hcx\" (UID: \"c0aec0a3-2014-4d88-a168-33b4b7048bb0\") " pod="openshift-image-registry/node-ca-57hcx" Jul 02 19:07:38.629535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618085 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-run-netns\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.629535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618102 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-registration-dir\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.629535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618128 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-run-k8s-cni-cncf-io\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.629535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618144 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-var-lib-cni-multus\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.629535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618207 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-cnibin\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.629535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618248 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-os-release\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618276 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618301 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5jqs\" (UniqueName: \"kubernetes.io/projected/fe2caa8f-210e-4e84-89bd-f4158314d8c8-kube-api-access-z5jqs\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618324 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fe5b83d0-db4b-4c20-9083-f693a6885494-cni-binary-copy\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618347 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-var-lib-cni-bin\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618371 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-system-cni-dir\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618416 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-multus-socket-dir-parent\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618443 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-hostroot\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618463 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-run-multus-certs\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618483 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtzkz\" (UniqueName: \"kubernetes.io/projected/fe5b83d0-db4b-4c20-9083-f693a6885494-kube-api-access-vtzkz\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.618978 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.619282 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.619291 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-q4lpx\"" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.619396 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.620636 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:38.620686 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.620953 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.621089 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.621145 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.622398 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.622467 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.622398 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xd8wh\"" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.622965 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.623340 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.624991 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.630131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.627321 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-28tn2\"" Jul 02 19:07:38.631348 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.627371 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 19:07:38.631348 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.627447 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 19:07:38.631348 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.627538 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jkjv7" Jul 02 19:07:38.631348 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.630559 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.631348 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.631188 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 19:07:38.631348 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.631224 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-992l8\"" Jul 02 19:07:38.631828 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.631452 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 19:07:38.632769 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.632286 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 19:07:38.632769 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.632280 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 19:02:38 +0000 UTC" deadline="2028-03-08 05:17:05.780929467 +0000 UTC" Jul 02 19:07:38.632769 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.632324 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14746h9m27.148607898s" Jul 02 19:07:38.632769 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.632357 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-161.ec2.internal" Jul 02 19:07:38.632769 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.632563 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 19:07:38.633428 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.633171 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dt5td\"" Jul 02 19:07:38.633925 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.633497 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 19:07:38.633925 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.633770 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:38.633925 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:38.633828 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.636621 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal"] Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.636701 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vqm46" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.639072 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.639113 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.639120 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wnwhj\"" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.642078 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-161.ec2.internal"] Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.642551 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.712228 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.718948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-var-lib-cni-bin\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719008 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-run-netns\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719045 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-var-lib-openvswitch\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719100 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-etc-openvswitch\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719126 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-var-lib-cni-bin\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719145 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-kubernetes\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719178 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-system-cni-dir\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719204 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-multus-socket-dir-parent\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719233 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtzkz\" (UniqueName: \"kubernetes.io/projected/fe5b83d0-db4b-4c20-9083-f693a6885494-kube-api-access-vtzkz\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719253 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-system-cni-dir\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719256 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-run-ovn-kubernetes\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719319 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-sysctl-d\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719363 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-multus-socket-dir-parent\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.733314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719376 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-sys-fs\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.734351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719421 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-etc-kubernetes\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.734351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719439 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-sys-fs\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.734351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719448 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5562\" (UniqueName: \"kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562\") pod \"network-check-target-9qvrm\" (UID: \"f43fc6d3-00d7-4153-aea4-423b1d40e67b\") " pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:38.734351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719475 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-etc-kubernetes\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.734351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719476 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/365022fe-6d5e-4e74-be92-b791e81692b2-cnibin\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.734351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719514 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/365022fe-6d5e-4e74-be92-b791e81692b2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.734351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719539 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-run\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.734351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719562 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs\") pod \"network-metrics-daemon-rjlz4\" (UID: \"04c1f8e7-1552-41f6-b1af-d4ba971e3f76\") " pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:38.734351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719587 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/798f8da0-e51d-47d2-b055-6456e8e9c1e1-konnectivity-ca\") pod \"konnectivity-agent-vqm46\" (UID: \"798f8da0-e51d-47d2-b055-6456e8e9c1e1\") " pod="kube-system/konnectivity-agent-vqm46" Jul 02 19:07:38.734351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719614 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-socket-dir\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.734351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719664 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-device-dir\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.734351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719691 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-multus-cni-dir\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.734351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719714 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3b53746b-995c-49ce-a305-0993bd5f666b-host-slash\") pod \"iptables-alerter-ncfg2\" (UID: \"3b53746b-995c-49ce-a305-0993bd5f666b\") " pod="openshift-network-operator/iptables-alerter-ncfg2" Jul 02 19:07:38.734351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719724 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-device-dir\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.734351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/365022fe-6d5e-4e74-be92-b791e81692b2-os-release\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.734351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719772 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/365022fe-6d5e-4e74-be92-b791e81692b2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.735257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719792 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-cni-netd\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.735257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719811 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.735257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719826 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c0aec0a3-2014-4d88-a168-33b4b7048bb0-host\") pod \"node-ca-57hcx\" (UID: \"c0aec0a3-2014-4d88-a168-33b4b7048bb0\") " pod="openshift-image-registry/node-ca-57hcx" Jul 02 19:07:38.735257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719846 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwzsk\" (UniqueName: \"kubernetes.io/projected/3b53746b-995c-49ce-a305-0993bd5f666b-kube-api-access-zwzsk\") pod \"iptables-alerter-ncfg2\" (UID: \"3b53746b-995c-49ce-a305-0993bd5f666b\") " pod="openshift-network-operator/iptables-alerter-ncfg2" Jul 02 19:07:38.735257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719876 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-multus-cni-dir\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.735257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719891 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c0aec0a3-2014-4d88-a168-33b4b7048bb0-host\") pod \"node-ca-57hcx\" (UID: \"c0aec0a3-2014-4d88-a168-33b4b7048bb0\") " pod="openshift-image-registry/node-ca-57hcx" Jul 02 19:07:38.735257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719928 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc-tmp-dir\") pod \"node-resolver-jkjv7\" (UID: \"8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc\") " pod="openshift-dns/node-resolver-jkjv7" Jul 02 19:07:38.735257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719938 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.735257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719954 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/365022fe-6d5e-4e74-be92-b791e81692b2-system-cni-dir\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.735257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719973 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9a0d9114-0c59-4c0d-8615-f98267f398b6-ovnkube-config\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.735257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.719991 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-lib-modules\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.735257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720019 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-run-k8s-cni-cncf-io\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.735257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720044 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-var-lib-cni-multus\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.735257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720061 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxrhc\" (UniqueName: \"kubernetes.io/projected/365022fe-6d5e-4e74-be92-b791e81692b2-kube-api-access-lxrhc\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.735257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720062 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-run-k8s-cni-cncf-io\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.735257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720077 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjqvl\" (UniqueName: \"kubernetes.io/projected/a70064a5-a7a7-42a3-ba39-d71b617e51a9-kube-api-access-hjqvl\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.735257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720104 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-var-lib-cni-multus\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.735961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720106 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/365022fe-6d5e-4e74-be92-b791e81692b2-cni-binary-copy\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.735961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720149 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-systemd-units\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.735961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720180 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-run-openvswitch\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.735961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720231 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-log-socket\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.735961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720256 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.735961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720290 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-sysctl-conf\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.735961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720315 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-sys\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.735961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720334 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a70064a5-a7a7-42a3-ba39-d71b617e51a9-tmp\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.735961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720348 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-node-log\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.735961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720368 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5jqs\" (UniqueName: \"kubernetes.io/projected/fe2caa8f-210e-4e84-89bd-f4158314d8c8-kube-api-access-z5jqs\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.735961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720446 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fe5b83d0-db4b-4c20-9083-f693a6885494-cni-binary-copy\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.735961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720563 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7bd39ca6-f010-4e22-b7b9-04bf2acece57-dbus\") pod \"global-pull-secret-syncer-p2dbb\" (UID: \"7bd39ca6-f010-4e22-b7b9-04bf2acece57\") " pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:38.735961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720597 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9a0d9114-0c59-4c0d-8615-f98267f398b6-env-overrides\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.735961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720639 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-host\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.735961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720681 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-hostroot\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.735961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720708 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-run-multus-certs\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.735961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720734 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-slash\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.736609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720758 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-run-systemd\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.736609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720780 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-run-multus-certs\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.736609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720790 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwshw\" (UniqueName: \"kubernetes.io/projected/c0aec0a3-2014-4d88-a168-33b4b7048bb0-kube-api-access-wwshw\") pod \"node-ca-57hcx\" (UID: \"c0aec0a3-2014-4d88-a168-33b4b7048bb0\") " pod="openshift-image-registry/node-ca-57hcx" Jul 02 19:07:38.736609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720823 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-var-lib-kubelet\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.736609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720855 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-multus-conf-dir\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.736609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720888 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3b53746b-995c-49ce-a305-0993bd5f666b-iptables-alerter-script\") pod \"iptables-alerter-ncfg2\" (UID: \"3b53746b-995c-49ce-a305-0993bd5f666b\") " pod="openshift-network-operator/iptables-alerter-ncfg2" Jul 02 19:07:38.736609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720902 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-multus-conf-dir\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.736609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720920 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-var-lib-kubelet\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.736609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720924 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f9rt\" (UniqueName: \"kubernetes.io/projected/8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc-kube-api-access-5f9rt\") pod \"node-resolver-jkjv7\" (UID: \"8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc\") " pod="openshift-dns/node-resolver-jkjv7" Jul 02 19:07:38.736609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.721705 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-socket-dir\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.736609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.720756 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-hostroot\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.736609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.721830 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/798f8da0-e51d-47d2-b055-6456e8e9c1e1-agent-certs\") pod \"konnectivity-agent-vqm46\" (UID: \"798f8da0-e51d-47d2-b055-6456e8e9c1e1\") " pod="kube-system/konnectivity-agent-vqm46" Jul 02 19:07:38.736609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.721870 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c0aec0a3-2014-4d88-a168-33b4b7048bb0-serviceca\") pod \"node-ca-57hcx\" (UID: \"c0aec0a3-2014-4d88-a168-33b4b7048bb0\") " pod="openshift-image-registry/node-ca-57hcx" Jul 02 19:07:38.736609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.721905 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fe5b83d0-db4b-4c20-9083-f693a6885494-multus-daemon-config\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.736609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.721949 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/365022fe-6d5e-4e74-be92-b791e81692b2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.736609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.721981 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-kubelet\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.736609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722014 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n66vp\" (UniqueName: \"kubernetes.io/projected/9a0d9114-0c59-4c0d-8615-f98267f398b6-kube-api-access-n66vp\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.737159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722040 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltskr\" (UniqueName: \"kubernetes.io/projected/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-kube-api-access-ltskr\") pod \"network-metrics-daemon-rjlz4\" (UID: \"04c1f8e7-1552-41f6-b1af-d4ba971e3f76\") " pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:38.737159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722075 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-run-netns\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.737159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722207 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-host-run-netns\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.737159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722357 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-run-ovn\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.737159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722445 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9a0d9114-0c59-4c0d-8615-f98267f398b6-ovnkube-script-lib\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.737159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722532 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-registration-dir\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.737159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722602 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-registration-dir\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.737159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722606 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret\") pod \"global-pull-secret-syncer-p2dbb\" (UID: \"7bd39ca6-f010-4e22-b7b9-04bf2acece57\") " pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:38.737159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc-hosts-file\") pod \"node-resolver-jkjv7\" (UID: \"8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc\") " pod="openshift-dns/node-resolver-jkjv7" Jul 02 19:07:38.737159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722684 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9a0d9114-0c59-4c0d-8615-f98267f398b6-ovn-node-metrics-cert\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.737159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722712 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-systemd\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.737159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722754 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-cnibin\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.737159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722788 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-os-release\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.737159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722812 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7bd39ca6-f010-4e22-b7b9-04bf2acece57-kubelet-config\") pod \"global-pull-secret-syncer-p2dbb\" (UID: \"7bd39ca6-f010-4e22-b7b9-04bf2acece57\") " pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:38.737159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722834 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-cni-bin\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.737159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722850 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-sysconfig\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.737159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722877 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-var-lib-kubelet\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.738169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722948 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-cnibin\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.738169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722967 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fe5b83d0-db4b-4c20-9083-f693a6885494-os-release\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.738169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.722997 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-tuned\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.738169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.723039 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.738169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.723074 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-modprobe-d\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.738169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.723209 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fe2caa8f-210e-4e84-89bd-f4158314d8c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.738169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.723270 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fe5b83d0-db4b-4c20-9083-f693a6885494-cni-binary-copy\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.738169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.723283 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fe5b83d0-db4b-4c20-9083-f693a6885494-multus-daemon-config\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.738169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.725819 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c0aec0a3-2014-4d88-a168-33b4b7048bb0-serviceca\") pod \"node-ca-57hcx\" (UID: \"c0aec0a3-2014-4d88-a168-33b4b7048bb0\") " pod="openshift-image-registry/node-ca-57hcx" Jul 02 19:07:38.738169 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:38.725917 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:07:38.738169 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:38.725935 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:07:38.738169 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:38.725948 2578 projected.go:196] Error preparing data for projected volume kube-api-access-q5562 for pod openshift-network-diagnostics/network-check-target-9qvrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:07:38.738169 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:38.726009 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562 podName:f43fc6d3-00d7-4153-aea4-423b1d40e67b nodeName:}" failed. No retries permitted until 2026-07-02 19:07:39.225993295 +0000 UTC m=+2.028808726 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-q5562" (UniqueName: "kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562") pod "network-check-target-9qvrm" (UID: "f43fc6d3-00d7-4153-aea4-423b1d40e67b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:07:38.738169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.726176 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 19:07:38.738169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.730309 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5jqs\" (UniqueName: \"kubernetes.io/projected/fe2caa8f-210e-4e84-89bd-f4158314d8c8-kube-api-access-z5jqs\") pod \"aws-ebs-csi-driver-node-jn7jc\" (UID: \"fe2caa8f-210e-4e84-89bd-f4158314d8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:38.738169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.730309 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwshw\" (UniqueName: \"kubernetes.io/projected/c0aec0a3-2014-4d88-a168-33b4b7048bb0-kube-api-access-wwshw\") pod \"node-ca-57hcx\" (UID: \"c0aec0a3-2014-4d88-a168-33b4b7048bb0\") " pod="openshift-image-registry/node-ca-57hcx" Jul 02 19:07:38.738169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.732009 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtzkz\" (UniqueName: \"kubernetes.io/projected/fe5b83d0-db4b-4c20-9083-f693a6885494-kube-api-access-vtzkz\") pod \"multus-vtngm\" (UID: \"fe5b83d0-db4b-4c20-9083-f693a6885494\") " pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.739771 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nhtc5" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.749929 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nhtc5" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.764055 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vtngm" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.824288 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3b53746b-995c-49ce-a305-0993bd5f666b-iptables-alerter-script\") pod \"iptables-alerter-ncfg2\" (UID: \"3b53746b-995c-49ce-a305-0993bd5f666b\") " pod="openshift-network-operator/iptables-alerter-ncfg2" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.824315 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f9rt\" (UniqueName: \"kubernetes.io/projected/8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc-kube-api-access-5f9rt\") pod \"node-resolver-jkjv7\" (UID: \"8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc\") " pod="openshift-dns/node-resolver-jkjv7" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.824331 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/798f8da0-e51d-47d2-b055-6456e8e9c1e1-agent-certs\") pod \"konnectivity-agent-vqm46\" (UID: \"798f8da0-e51d-47d2-b055-6456e8e9c1e1\") " pod="kube-system/konnectivity-agent-vqm46" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.824588 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/365022fe-6d5e-4e74-be92-b791e81692b2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.824724 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-kubelet\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.824752 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n66vp\" (UniqueName: \"kubernetes.io/projected/9a0d9114-0c59-4c0d-8615-f98267f398b6-kube-api-access-n66vp\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.824778 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltskr\" (UniqueName: \"kubernetes.io/projected/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-kube-api-access-ltskr\") pod \"network-metrics-daemon-rjlz4\" (UID: \"04c1f8e7-1552-41f6-b1af-d4ba971e3f76\") " pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.824809 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-run-ovn\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.824835 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9a0d9114-0c59-4c0d-8615-f98267f398b6-ovnkube-script-lib\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.824865 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret\") pod \"global-pull-secret-syncer-p2dbb\" (UID: \"7bd39ca6-f010-4e22-b7b9-04bf2acece57\") " pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.824890 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc-hosts-file\") pod \"node-resolver-jkjv7\" (UID: \"8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc\") " pod="openshift-dns/node-resolver-jkjv7" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.824915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9a0d9114-0c59-4c0d-8615-f98267f398b6-ovn-node-metrics-cert\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.824927 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-run-ovn\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.824941 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-systemd\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.824969 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7bd39ca6-f010-4e22-b7b9-04bf2acece57-kubelet-config\") pod \"global-pull-secret-syncer-p2dbb\" (UID: \"7bd39ca6-f010-4e22-b7b9-04bf2acece57\") " pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:38.835415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.824995 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-cni-bin\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.836951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.824999 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3b53746b-995c-49ce-a305-0993bd5f666b-iptables-alerter-script\") pod \"iptables-alerter-ncfg2\" (UID: \"3b53746b-995c-49ce-a305-0993bd5f666b\") " pod="openshift-network-operator/iptables-alerter-ncfg2" Jul 02 19:07:38.836951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.825014 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-sysconfig\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.836951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.825033 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-var-lib-kubelet\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.836951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.825063 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-kubelet\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.836951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.825124 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7bd39ca6-f010-4e22-b7b9-04bf2acece57-kubelet-config\") pod \"global-pull-secret-syncer-p2dbb\" (UID: \"7bd39ca6-f010-4e22-b7b9-04bf2acece57\") " pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:38.836951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.825176 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-systemd\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.836951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.825224 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc-hosts-file\") pod \"node-resolver-jkjv7\" (UID: \"8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc\") " pod="openshift-dns/node-resolver-jkjv7" Jul 02 19:07:38.836951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.825268 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-sysconfig\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.836951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.825306 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-cni-bin\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.836951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.825351 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-var-lib-kubelet\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.836951 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:38.825575 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:07:38.836951 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:38.825636 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret podName:7bd39ca6-f010-4e22-b7b9-04bf2acece57 nodeName:}" failed. No retries permitted until 2026-07-02 19:07:39.325619749 +0000 UTC m=+2.128435180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret") pod "global-pull-secret-syncer-p2dbb" (UID: "7bd39ca6-f010-4e22-b7b9-04bf2acece57") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:07:38.836951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.834242 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-tuned\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.836951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.834294 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-modprobe-d\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.836951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.834320 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-run-netns\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.836951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.834353 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-var-lib-openvswitch\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.836951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.834369 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-etc-openvswitch\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.841412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.834387 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-kubernetes\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.841412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.834422 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-run-ovn-kubernetes\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.841412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.834438 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-sysctl-d\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.841412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.834496 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-etc-openvswitch\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.841412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.834589 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-var-lib-openvswitch\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.841412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.834634 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-kubernetes\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.841412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.834666 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-run-ovn-kubernetes\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.841412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.834674 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9a0d9114-0c59-4c0d-8615-f98267f398b6-ovnkube-script-lib\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.841412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.834748 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-sysctl-d\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.841412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.834436 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-modprobe-d\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.841412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835190 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-run-netns\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.841412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835197 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/365022fe-6d5e-4e74-be92-b791e81692b2-cnibin\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.841412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835229 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/365022fe-6d5e-4e74-be92-b791e81692b2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.841412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835256 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-run\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.841412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835280 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs\") pod \"network-metrics-daemon-rjlz4\" (UID: \"04c1f8e7-1552-41f6-b1af-d4ba971e3f76\") " pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:38.841412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835325 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/798f8da0-e51d-47d2-b055-6456e8e9c1e1-konnectivity-ca\") pod \"konnectivity-agent-vqm46\" (UID: \"798f8da0-e51d-47d2-b055-6456e8e9c1e1\") " pod="kube-system/konnectivity-agent-vqm46" Jul 02 19:07:38.841412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835354 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3b53746b-995c-49ce-a305-0993bd5f666b-host-slash\") pod \"iptables-alerter-ncfg2\" (UID: \"3b53746b-995c-49ce-a305-0993bd5f666b\") " pod="openshift-network-operator/iptables-alerter-ncfg2" Jul 02 19:07:38.846189 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835379 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/365022fe-6d5e-4e74-be92-b791e81692b2-os-release\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.846189 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835406 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/365022fe-6d5e-4e74-be92-b791e81692b2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.846189 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835432 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-cni-netd\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.846189 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835461 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-run\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.846189 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835461 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwzsk\" (UniqueName: \"kubernetes.io/projected/3b53746b-995c-49ce-a305-0993bd5f666b-kube-api-access-zwzsk\") pod \"iptables-alerter-ncfg2\" (UID: \"3b53746b-995c-49ce-a305-0993bd5f666b\") " pod="openshift-network-operator/iptables-alerter-ncfg2" Jul 02 19:07:38.846189 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835525 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/365022fe-6d5e-4e74-be92-b791e81692b2-cnibin\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.846189 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:38.835561 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:07:38.846189 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:38.835608 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs podName:04c1f8e7-1552-41f6-b1af-d4ba971e3f76 nodeName:}" failed. No retries permitted until 2026-07-02 19:07:39.33559275 +0000 UTC m=+2.138408180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs") pod "network-metrics-daemon-rjlz4" (UID: "04c1f8e7-1552-41f6-b1af-d4ba971e3f76") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:07:38.846189 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835839 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/365022fe-6d5e-4e74-be92-b791e81692b2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.846189 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835880 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc-tmp-dir\") pod \"node-resolver-jkjv7\" (UID: \"8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc\") " pod="openshift-dns/node-resolver-jkjv7" Jul 02 19:07:38.846189 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835911 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/365022fe-6d5e-4e74-be92-b791e81692b2-system-cni-dir\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.846189 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835941 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9a0d9114-0c59-4c0d-8615-f98267f398b6-ovnkube-config\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.846189 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835967 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-lib-modules\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.846189 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.835995 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxrhc\" (UniqueName: \"kubernetes.io/projected/365022fe-6d5e-4e74-be92-b791e81692b2-kube-api-access-lxrhc\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.846189 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836041 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjqvl\" (UniqueName: \"kubernetes.io/projected/a70064a5-a7a7-42a3-ba39-d71b617e51a9-kube-api-access-hjqvl\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.846189 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836069 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/365022fe-6d5e-4e74-be92-b791e81692b2-cni-binary-copy\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.846189 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836095 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-systemd-units\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.846894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836122 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-run-openvswitch\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.846894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836148 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-log-socket\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.846894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836157 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/365022fe-6d5e-4e74-be92-b791e81692b2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.846894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836178 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.846894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836214 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-sysctl-conf\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.846894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836240 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-sys\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.846894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836265 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a70064a5-a7a7-42a3-ba39-d71b617e51a9-tmp\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.846894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836290 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-node-log\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.846894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836326 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/798f8da0-e51d-47d2-b055-6456e8e9c1e1-konnectivity-ca\") pod \"konnectivity-agent-vqm46\" (UID: \"798f8da0-e51d-47d2-b055-6456e8e9c1e1\") " pod="kube-system/konnectivity-agent-vqm46" Jul 02 19:07:38.846894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836334 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7bd39ca6-f010-4e22-b7b9-04bf2acece57-dbus\") pod \"global-pull-secret-syncer-p2dbb\" (UID: \"7bd39ca6-f010-4e22-b7b9-04bf2acece57\") " pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:38.846894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836360 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9a0d9114-0c59-4c0d-8615-f98267f398b6-env-overrides\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.846894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836387 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-cni-netd\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.846894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836386 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-host\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.846894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836436 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-slash\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.846894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836442 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3b53746b-995c-49ce-a305-0993bd5f666b-host-slash\") pod \"iptables-alerter-ncfg2\" (UID: \"3b53746b-995c-49ce-a305-0993bd5f666b\") " pod="openshift-network-operator/iptables-alerter-ncfg2" Jul 02 19:07:38.846894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836461 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-run-systemd\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.846894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836531 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/365022fe-6d5e-4e74-be92-b791e81692b2-os-release\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836561 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-run-systemd\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836576 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-log-socket\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836624 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836778 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-sysctl-conf\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836818 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/365022fe-6d5e-4e74-be92-b791e81692b2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836843 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7bd39ca6-f010-4e22-b7b9-04bf2acece57-dbus\") pod \"global-pull-secret-syncer-p2dbb\" (UID: \"7bd39ca6-f010-4e22-b7b9-04bf2acece57\") " pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836894 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-host-slash\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836905 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-host\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836943 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-node-log\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.836991 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-sys\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.837348 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-systemd-units\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.837432 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a0d9114-0c59-4c0d-8615-f98267f398b6-run-openvswitch\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.837544 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a70064a5-a7a7-42a3-ba39-d71b617e51a9-lib-modules\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.837729 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc-tmp-dir\") pod \"node-resolver-jkjv7\" (UID: \"8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc\") " pod="openshift-dns/node-resolver-jkjv7" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.837863 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/365022fe-6d5e-4e74-be92-b791e81692b2-system-cni-dir\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.837919 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/365022fe-6d5e-4e74-be92-b791e81692b2-cni-binary-copy\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.838079 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a70064a5-a7a7-42a3-ba39-d71b617e51a9-etc-tuned\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.847588 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.838171 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f9rt\" (UniqueName: \"kubernetes.io/projected/8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc-kube-api-access-5f9rt\") pod \"node-resolver-jkjv7\" (UID: \"8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc\") " pod="openshift-dns/node-resolver-jkjv7" Jul 02 19:07:38.849231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.838438 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/798f8da0-e51d-47d2-b055-6456e8e9c1e1-agent-certs\") pod \"konnectivity-agent-vqm46\" (UID: \"798f8da0-e51d-47d2-b055-6456e8e9c1e1\") " pod="kube-system/konnectivity-agent-vqm46" Jul 02 19:07:38.849231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.838101 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9a0d9114-0c59-4c0d-8615-f98267f398b6-ovn-node-metrics-cert\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.849231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.838759 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9a0d9114-0c59-4c0d-8615-f98267f398b6-env-overrides\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.849231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.839084 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9a0d9114-0c59-4c0d-8615-f98267f398b6-ovnkube-config\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.849231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.841118 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n66vp\" (UniqueName: \"kubernetes.io/projected/9a0d9114-0c59-4c0d-8615-f98267f398b6-kube-api-access-n66vp\") pod \"ovnkube-node-7h54r\" (UID: \"9a0d9114-0c59-4c0d-8615-f98267f398b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:38.849231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.842295 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltskr\" (UniqueName: \"kubernetes.io/projected/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-kube-api-access-ltskr\") pod \"network-metrics-daemon-rjlz4\" (UID: \"04c1f8e7-1552-41f6-b1af-d4ba971e3f76\") " pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:38.849231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.845615 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwzsk\" (UniqueName: \"kubernetes.io/projected/3b53746b-995c-49ce-a305-0993bd5f666b-kube-api-access-zwzsk\") pod \"iptables-alerter-ncfg2\" (UID: \"3b53746b-995c-49ce-a305-0993bd5f666b\") " pod="openshift-network-operator/iptables-alerter-ncfg2" Jul 02 19:07:38.966033 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.840316 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a70064a5-a7a7-42a3-ba39-d71b617e51a9-tmp\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.966033 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.860492 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:07:38.966033 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.860737 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjqvl\" (UniqueName: \"kubernetes.io/projected/a70064a5-a7a7-42a3-ba39-d71b617e51a9-kube-api-access-hjqvl\") pod \"tuned-l5h6q\" (UID: \"a70064a5-a7a7-42a3-ba39-d71b617e51a9\") " pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.966033 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.862718 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxrhc\" (UniqueName: \"kubernetes.io/projected/365022fe-6d5e-4e74-be92-b791e81692b2-kube-api-access-lxrhc\") pod \"multus-additional-cni-plugins-prfg6\" (UID: \"365022fe-6d5e-4e74-be92-b791e81692b2\") " pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.966033 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.885240 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jkjv7" Jul 02 19:07:38.966033 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.893737 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" Jul 02 19:07:38.966033 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.899458 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-prfg6" Jul 02 19:07:38.966033 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.901163 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vqm46" Jul 02 19:07:38.966033 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:38.917047 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05f91772ec7f4ad16d2e21378d51cdee.slice/crio-03da8d5d1158a6ca62a6cb5f3da9f6a2f9f569fa430ea021b77297abd8e9fe08 WatchSource:0}: Error finding container 03da8d5d1158a6ca62a6cb5f3da9f6a2f9f569fa430ea021b77297abd8e9fe08: Status 404 returned error can't find the container with id 03da8d5d1158a6ca62a6cb5f3da9f6a2f9f569fa430ea021b77297abd8e9fe08 Jul 02 19:07:38.966033 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:38.917456 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod798f8da0_e51d_47d2_b055_6456e8e9c1e1.slice/crio-10bdf8c69881a995fa50ab92f289dc0cb246312d975aec0f66a484a59a6bd695 WatchSource:0}: Error finding container 10bdf8c69881a995fa50ab92f289dc0cb246312d975aec0f66a484a59a6bd695: Status 404 returned error can't find the container with id 10bdf8c69881a995fa50ab92f289dc0cb246312d975aec0f66a484a59a6bd695 Jul 02 19:07:38.966033 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:38.918235 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda70064a5_a7a7_42a3_ba39_d71b617e51a9.slice/crio-27e77353fa120896cdb897ba0efeb10823bb722e2c0a8b1b648ec61b726e9d78 WatchSource:0}: Error finding container 27e77353fa120896cdb897ba0efeb10823bb722e2c0a8b1b648ec61b726e9d78: Status 404 returned error can't find the container with id 27e77353fa120896cdb897ba0efeb10823bb722e2c0a8b1b648ec61b726e9d78 Jul 02 19:07:38.966033 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:38.919092 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3d1df1679904c800dfb191bda1f933a.slice/crio-80b16fe10be280778f63eba5d827e8991b84f9ac155231d42a84c762951607d8 WatchSource:0}: Error finding container 80b16fe10be280778f63eba5d827e8991b84f9ac155231d42a84c762951607d8: Status 404 returned error can't find the container with id 80b16fe10be280778f63eba5d827e8991b84f9ac155231d42a84c762951607d8 Jul 02 19:07:38.966033 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:38.920062 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod365022fe_6d5e_4e74_be92_b791e81692b2.slice/crio-22fcb23b920a7cd0329e90bb1886e426bafac0135d709125fb471b1c5ac23f95 WatchSource:0}: Error finding container 22fcb23b920a7cd0329e90bb1886e426bafac0135d709125fb471b1c5ac23f95: Status 404 returned error can't find the container with id 22fcb23b920a7cd0329e90bb1886e426bafac0135d709125fb471b1c5ac23f95 Jul 02 19:07:38.966033 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:38.920856 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e01d2a6_a6aa_42e3_94b5_164e28d5a3cc.slice/crio-a1e09961bc778535a685217a5cacdc87d510e2634529d5561fb147dc85b0fa1d WatchSource:0}: Error finding container a1e09961bc778535a685217a5cacdc87d510e2634529d5561fb147dc85b0fa1d: Status 404 returned error can't find the container with id a1e09961bc778535a685217a5cacdc87d510e2634529d5561fb147dc85b0fa1d Jul 02 19:07:38.966033 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.929809 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 19:07:38.966033 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.958861 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:07:39.068298 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:38.987117 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" Jul 02 19:07:39.068298 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:38.999230 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe2caa8f_210e_4e84_89bd_f4158314d8c8.slice/crio-80c041af6780a7dced38d34315955e30c5cf4b3cc78dc7d6631959e807248ddf WatchSource:0}: Error finding container 80c041af6780a7dced38d34315955e30c5cf4b3cc78dc7d6631959e807248ddf: Status 404 returned error can't find the container with id 80c041af6780a7dced38d34315955e30c5cf4b3cc78dc7d6631959e807248ddf Jul 02 19:07:39.068298 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.023533 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-57hcx" Jul 02 19:07:39.068298 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:39.034659 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0aec0a3_2014_4d88_a168_33b4b7048bb0.slice/crio-93e68626d0802cc6bfabfa612eda0879a4286ddd7a6927244d5c939e577e0951 WatchSource:0}: Error finding container 93e68626d0802cc6bfabfa612eda0879a4286ddd7a6927244d5c939e577e0951: Status 404 returned error can't find the container with id 93e68626d0802cc6bfabfa612eda0879a4286ddd7a6927244d5c939e577e0951 Jul 02 19:07:39.169467 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.099719 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ncfg2" Jul 02 19:07:39.169467 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.104999 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:07:39.169467 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:39.113415 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a0d9114_0c59_4c0d_8615_f98267f398b6.slice/crio-7b2d4553d77af48e50815e787ff9ce7eb459244ec70b47fe171db857754b8a2d WatchSource:0}: Error finding container 7b2d4553d77af48e50815e787ff9ce7eb459244ec70b47fe171db857754b8a2d: Status 404 returned error can't find the container with id 7b2d4553d77af48e50815e787ff9ce7eb459244ec70b47fe171db857754b8a2d Jul 02 19:07:39.169467 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:39.122423 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b53746b_995c_49ce_a305_0993bd5f666b.slice/crio-17884def6e57bfeefd64887acf09a4b05403338cd77bcd67bc7fe0b9d41c9c3f WatchSource:0}: Error finding container 17884def6e57bfeefd64887acf09a4b05403338cd77bcd67bc7fe0b9d41c9c3f: Status 404 returned error can't find the container with id 17884def6e57bfeefd64887acf09a4b05403338cd77bcd67bc7fe0b9d41c9c3f Jul 02 19:07:39.292357 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:07:39.210063 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe5b83d0_db4b_4c20_9083_f693a6885494.slice/crio-dc9c0e2732d8e4837500ecadc417c8127ca0ab726382845410d3361fd92b4370 WatchSource:0}: Error finding container dc9c0e2732d8e4837500ecadc417c8127ca0ab726382845410d3361fd92b4370: Status 404 returned error can't find the container with id dc9c0e2732d8e4837500ecadc417c8127ca0ab726382845410d3361fd92b4370 Jul 02 19:07:39.292357 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.241282 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5562\" (UniqueName: \"kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562\") pod \"network-check-target-9qvrm\" (UID: \"f43fc6d3-00d7-4153-aea4-423b1d40e67b\") " pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:39.292357 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:39.241405 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:07:39.292357 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:39.241439 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:07:39.292357 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:39.241449 2578 projected.go:196] Error preparing data for projected volume kube-api-access-q5562 for pod openshift-network-diagnostics/network-check-target-9qvrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:07:39.292357 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:39.241493 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562 podName:f43fc6d3-00d7-4153-aea4-423b1d40e67b nodeName:}" failed. No retries permitted until 2026-07-02 19:07:40.241478198 +0000 UTC m=+3.044293629 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-q5562" (UniqueName: "kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562") pod "network-check-target-9qvrm" (UID: "f43fc6d3-00d7-4153-aea4-423b1d40e67b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:07:39.403048 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.341680 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs\") pod \"network-metrics-daemon-rjlz4\" (UID: \"04c1f8e7-1552-41f6-b1af-d4ba971e3f76\") " pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:39.403048 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.341742 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret\") pod \"global-pull-secret-syncer-p2dbb\" (UID: \"7bd39ca6-f010-4e22-b7b9-04bf2acece57\") " pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:39.403048 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:39.341834 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:07:39.403048 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:39.341889 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret podName:7bd39ca6-f010-4e22-b7b9-04bf2acece57 nodeName:}" failed. No retries permitted until 2026-07-02 19:07:40.341874566 +0000 UTC m=+3.144690000 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret") pod "global-pull-secret-syncer-p2dbb" (UID: "7bd39ca6-f010-4e22-b7b9-04bf2acece57") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:07:39.403048 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:39.341834 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:07:39.403048 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:39.341975 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs podName:04c1f8e7-1552-41f6-b1af-d4ba971e3f76 nodeName:}" failed. No retries permitted until 2026-07-02 19:07:40.34195994 +0000 UTC m=+3.144775375 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs") pod "network-metrics-daemon-rjlz4" (UID: "04c1f8e7-1552-41f6-b1af-d4ba971e3f76") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:07:39.658306 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.658120 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:07:39.723106 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.722545 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:39.723106 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.722559 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:39.723106 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:39.722676 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:07:39.723106 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:39.722761 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:07:39.727512 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.727400 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" event={"ID":"fe2caa8f-210e-4e84-89bd-f4158314d8c8","Type":"ContainerStarted","Data":"80c041af6780a7dced38d34315955e30c5cf4b3cc78dc7d6631959e807248ddf"} Jul 02 19:07:39.739554 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.737728 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prfg6" event={"ID":"365022fe-6d5e-4e74-be92-b791e81692b2","Type":"ContainerStarted","Data":"22fcb23b920a7cd0329e90bb1886e426bafac0135d709125fb471b1c5ac23f95"} Jul 02 19:07:39.745619 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.745589 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" event={"ID":"a70064a5-a7a7-42a3-ba39-d71b617e51a9","Type":"ContainerStarted","Data":"27e77353fa120896cdb897ba0efeb10823bb722e2c0a8b1b648ec61b726e9d78"} Jul 02 19:07:39.748759 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.748690 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-161.ec2.internal" event={"ID":"b3d1df1679904c800dfb191bda1f933a","Type":"ContainerStarted","Data":"80b16fe10be280778f63eba5d827e8991b84f9ac155231d42a84c762951607d8"} Jul 02 19:07:39.753755 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.753727 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jkjv7" event={"ID":"8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc","Type":"ContainerStarted","Data":"a1e09961bc778535a685217a5cacdc87d510e2634529d5561fb147dc85b0fa1d"} Jul 02 19:07:39.753896 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.753843 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 19:02:39 +0000 UTC" deadline="2027-11-28 22:12:14.980883621 +0000 UTC" Jul 02 19:07:39.753957 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.753899 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12339h4m35.226988158s" Jul 02 19:07:39.762532 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.762504 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vtngm" event={"ID":"fe5b83d0-db4b-4c20-9083-f693a6885494","Type":"ContainerStarted","Data":"dc9c0e2732d8e4837500ecadc417c8127ca0ab726382845410d3361fd92b4370"} Jul 02 19:07:39.765796 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.765428 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vqm46" event={"ID":"798f8da0-e51d-47d2-b055-6456e8e9c1e1","Type":"ContainerStarted","Data":"10bdf8c69881a995fa50ab92f289dc0cb246312d975aec0f66a484a59a6bd695"} Jul 02 19:07:39.779687 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.779656 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal" event={"ID":"05f91772ec7f4ad16d2e21378d51cdee","Type":"ContainerStarted","Data":"03da8d5d1158a6ca62a6cb5f3da9f6a2f9f569fa430ea021b77297abd8e9fe08"} Jul 02 19:07:39.786668 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.786543 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" event={"ID":"9a0d9114-0c59-4c0d-8615-f98267f398b6","Type":"ContainerStarted","Data":"7b2d4553d77af48e50815e787ff9ce7eb459244ec70b47fe171db857754b8a2d"} Jul 02 19:07:39.793912 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.793775 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ncfg2" event={"ID":"3b53746b-995c-49ce-a305-0993bd5f666b","Type":"ContainerStarted","Data":"17884def6e57bfeefd64887acf09a4b05403338cd77bcd67bc7fe0b9d41c9c3f"} Jul 02 19:07:39.808943 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:39.808868 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-57hcx" event={"ID":"c0aec0a3-2014-4d88-a168-33b4b7048bb0","Type":"ContainerStarted","Data":"93e68626d0802cc6bfabfa612eda0879a4286ddd7a6927244d5c939e577e0951"} Jul 02 19:07:40.248868 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:40.248829 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5562\" (UniqueName: \"kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562\") pod \"network-check-target-9qvrm\" (UID: \"f43fc6d3-00d7-4153-aea4-423b1d40e67b\") " pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:40.249066 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:40.249024 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:07:40.249066 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:40.249046 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:07:40.249066 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:40.249060 2578 projected.go:196] Error preparing data for projected volume kube-api-access-q5562 for pod openshift-network-diagnostics/network-check-target-9qvrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:07:40.249247 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:40.249118 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562 podName:f43fc6d3-00d7-4153-aea4-423b1d40e67b nodeName:}" failed. No retries permitted until 2026-07-02 19:07:42.249100102 +0000 UTC m=+5.051915534 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-q5562" (UniqueName: "kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562") pod "network-check-target-9qvrm" (UID: "f43fc6d3-00d7-4153-aea4-423b1d40e67b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:07:40.350034 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:40.349998 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs\") pod \"network-metrics-daemon-rjlz4\" (UID: \"04c1f8e7-1552-41f6-b1af-d4ba971e3f76\") " pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:40.350207 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:40.350077 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret\") pod \"global-pull-secret-syncer-p2dbb\" (UID: \"7bd39ca6-f010-4e22-b7b9-04bf2acece57\") " pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:40.350288 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:40.350204 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:07:40.350288 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:40.350266 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret podName:7bd39ca6-f010-4e22-b7b9-04bf2acece57 nodeName:}" failed. No retries permitted until 2026-07-02 19:07:42.350247738 +0000 UTC m=+5.153063174 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret") pod "global-pull-secret-syncer-p2dbb" (UID: "7bd39ca6-f010-4e22-b7b9-04bf2acece57") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:07:40.350402 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:40.350328 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:07:40.350402 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:40.350365 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs podName:04c1f8e7-1552-41f6-b1af-d4ba971e3f76 nodeName:}" failed. No retries permitted until 2026-07-02 19:07:42.350351117 +0000 UTC m=+5.153166547 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs") pod "network-metrics-daemon-rjlz4" (UID: "04c1f8e7-1552-41f6-b1af-d4ba971e3f76") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:07:40.721368 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:40.721331 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:40.721954 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:40.721497 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:07:41.724230 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:41.724200 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:41.724710 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:41.724338 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:07:41.724844 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:41.724786 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:41.724900 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:41.724886 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:07:42.264374 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:42.264335 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5562\" (UniqueName: \"kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562\") pod \"network-check-target-9qvrm\" (UID: \"f43fc6d3-00d7-4153-aea4-423b1d40e67b\") " pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:42.264584 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:42.264525 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:07:42.264584 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:42.264545 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:07:42.264584 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:42.264559 2578 projected.go:196] Error preparing data for projected volume kube-api-access-q5562 for pod openshift-network-diagnostics/network-check-target-9qvrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:07:42.264761 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:42.264615 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562 podName:f43fc6d3-00d7-4153-aea4-423b1d40e67b nodeName:}" failed. No retries permitted until 2026-07-02 19:07:46.264597138 +0000 UTC m=+9.067412590 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-q5562" (UniqueName: "kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562") pod "network-check-target-9qvrm" (UID: "f43fc6d3-00d7-4153-aea4-423b1d40e67b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:07:42.365730 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:42.365690 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret\") pod \"global-pull-secret-syncer-p2dbb\" (UID: \"7bd39ca6-f010-4e22-b7b9-04bf2acece57\") " pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:42.365925 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:42.365774 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs\") pod \"network-metrics-daemon-rjlz4\" (UID: \"04c1f8e7-1552-41f6-b1af-d4ba971e3f76\") " pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:42.365989 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:42.365943 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:07:42.366046 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:42.366011 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs podName:04c1f8e7-1552-41f6-b1af-d4ba971e3f76 nodeName:}" failed. No retries permitted until 2026-07-02 19:07:46.36599014 +0000 UTC m=+9.168805574 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs") pod "network-metrics-daemon-rjlz4" (UID: "04c1f8e7-1552-41f6-b1af-d4ba971e3f76") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:07:42.366443 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:42.366424 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:07:42.366525 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:42.366499 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret podName:7bd39ca6-f010-4e22-b7b9-04bf2acece57 nodeName:}" failed. No retries permitted until 2026-07-02 19:07:46.366467225 +0000 UTC m=+9.169282661 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret") pod "global-pull-secret-syncer-p2dbb" (UID: "7bd39ca6-f010-4e22-b7b9-04bf2acece57") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:07:42.722407 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:42.721898 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:42.722407 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:42.722044 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:07:43.722645 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:43.722609 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:43.723154 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:43.722751 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:07:43.723154 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:43.723129 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:43.723253 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:43.723228 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:07:44.720918 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:44.720881 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:44.721115 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:44.721042 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:07:45.722020 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:45.721986 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:45.722559 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:45.722034 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:45.722559 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:45.722128 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:07:45.722559 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:45.722213 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:07:46.301075 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:46.301041 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5562\" (UniqueName: \"kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562\") pod \"network-check-target-9qvrm\" (UID: \"f43fc6d3-00d7-4153-aea4-423b1d40e67b\") " pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:46.301273 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:46.301244 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:07:46.301273 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:46.301270 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:07:46.301384 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:46.301285 2578 projected.go:196] Error preparing data for projected volume kube-api-access-q5562 for pod openshift-network-diagnostics/network-check-target-9qvrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:07:46.301384 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:46.301349 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562 podName:f43fc6d3-00d7-4153-aea4-423b1d40e67b nodeName:}" failed. No retries permitted until 2026-07-02 19:07:54.301330368 +0000 UTC m=+17.104145812 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-q5562" (UniqueName: "kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562") pod "network-check-target-9qvrm" (UID: "f43fc6d3-00d7-4153-aea4-423b1d40e67b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:07:46.402377 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:46.402290 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret\") pod \"global-pull-secret-syncer-p2dbb\" (UID: \"7bd39ca6-f010-4e22-b7b9-04bf2acece57\") " pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:46.402544 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:46.402458 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:07:46.402544 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:46.402525 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret podName:7bd39ca6-f010-4e22-b7b9-04bf2acece57 nodeName:}" failed. No retries permitted until 2026-07-02 19:07:54.402506564 +0000 UTC m=+17.205321996 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret") pod "global-pull-secret-syncer-p2dbb" (UID: "7bd39ca6-f010-4e22-b7b9-04bf2acece57") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:07:46.402749 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:46.402722 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs\") pod \"network-metrics-daemon-rjlz4\" (UID: \"04c1f8e7-1552-41f6-b1af-d4ba971e3f76\") " pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:46.402892 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:46.402872 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:07:46.402972 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:46.402943 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs podName:04c1f8e7-1552-41f6-b1af-d4ba971e3f76 nodeName:}" failed. No retries permitted until 2026-07-02 19:07:54.402924746 +0000 UTC m=+17.205740201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs") pod "network-metrics-daemon-rjlz4" (UID: "04c1f8e7-1552-41f6-b1af-d4ba971e3f76") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:07:46.721196 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:46.721112 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:46.721372 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:46.721258 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:07:47.722513 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:47.722479 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:47.723063 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:47.722604 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:07:47.723063 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:47.722492 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:47.723063 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:47.723003 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:07:48.720919 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:48.720890 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:48.721093 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:48.721023 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:07:49.723243 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:49.723206 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:49.723667 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:49.723336 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:07:49.723758 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:49.723741 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:49.723855 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:49.723836 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:07:50.721297 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:50.721260 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:50.721480 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:50.721464 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:07:51.720981 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:51.720958 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:51.721352 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:51.720962 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:51.721352 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:51.721083 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:07:51.721352 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:51.721141 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:07:52.721138 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:52.721100 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:52.721560 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:52.721223 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:07:53.721565 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:53.721534 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:53.721565 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:53.721550 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:53.722068 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:53.721664 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:07:53.722068 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:53.721742 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:07:54.364046 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:54.364009 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5562\" (UniqueName: \"kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562\") pod \"network-check-target-9qvrm\" (UID: \"f43fc6d3-00d7-4153-aea4-423b1d40e67b\") " pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:54.364265 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:54.364182 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:07:54.364265 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:54.364203 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:07:54.364265 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:54.364213 2578 projected.go:196] Error preparing data for projected volume kube-api-access-q5562 for pod openshift-network-diagnostics/network-check-target-9qvrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:07:54.364265 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:54.364265 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562 podName:f43fc6d3-00d7-4153-aea4-423b1d40e67b nodeName:}" failed. No retries permitted until 2026-07-02 19:08:10.364251464 +0000 UTC m=+33.167066915 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-q5562" (UniqueName: "kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562") pod "network-check-target-9qvrm" (UID: "f43fc6d3-00d7-4153-aea4-423b1d40e67b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:07:54.464894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:54.464858 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret\") pod \"global-pull-secret-syncer-p2dbb\" (UID: \"7bd39ca6-f010-4e22-b7b9-04bf2acece57\") " pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:54.465058 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:54.464924 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs\") pod \"network-metrics-daemon-rjlz4\" (UID: \"04c1f8e7-1552-41f6-b1af-d4ba971e3f76\") " pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:54.465058 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:54.465032 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:07:54.465175 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:54.465108 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret podName:7bd39ca6-f010-4e22-b7b9-04bf2acece57 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:10.465088251 +0000 UTC m=+33.267903686 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret") pod "global-pull-secret-syncer-p2dbb" (UID: "7bd39ca6-f010-4e22-b7b9-04bf2acece57") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:07:54.465175 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:54.465032 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:07:54.465296 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:54.465195 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs podName:04c1f8e7-1552-41f6-b1af-d4ba971e3f76 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:10.465176996 +0000 UTC m=+33.267992425 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs") pod "network-metrics-daemon-rjlz4" (UID: "04c1f8e7-1552-41f6-b1af-d4ba971e3f76") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:07:54.721604 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:54.721533 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:54.721975 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:54.721631 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:07:55.721073 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:55.721034 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:55.721262 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:55.721169 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:07:55.721262 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:55.721231 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:55.721373 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:55.721347 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:07:56.720835 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:56.720802 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:56.721323 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:56.720932 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:07:57.722208 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:57.722174 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:57.722674 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:57.722282 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:07:57.722674 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:57.722360 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:57.722674 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:57.722474 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:07:58.721814 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:58.721777 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:07:58.721991 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:58.721917 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:07:59.721731 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:59.721579 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:07:59.722142 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:59.721762 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:07:59.722142 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:59.721805 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:07:59.722142 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:07:59.721846 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:07:59.853674 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:59.853599 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" event={"ID":"9a0d9114-0c59-4c0d-8615-f98267f398b6","Type":"ContainerStarted","Data":"8bfab389011506d39c637f021f345a53b076feed5aa61aeb173cb664dfd8bac9"} Jul 02 19:07:59.854890 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:59.854868 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" event={"ID":"a70064a5-a7a7-42a3-ba39-d71b617e51a9","Type":"ContainerStarted","Data":"b6fad7a71c185279bb413c5d7354d97f8deae763bca47c795888012b4d423a63"} Jul 02 19:07:59.857558 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:59.857532 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-161.ec2.internal" event={"ID":"b3d1df1679904c800dfb191bda1f933a","Type":"ContainerStarted","Data":"7273ab2c534144901f47e52cc7b1c855c810ba161bab69856619348f58365537"} Jul 02 19:07:59.858993 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:59.858973 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vtngm" event={"ID":"fe5b83d0-db4b-4c20-9083-f693a6885494","Type":"ContainerStarted","Data":"98ca6e2aee2b57862646756f890f7d407aecf8c648b59dce1b41056edfdfba0b"} Jul 02 19:07:59.873547 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:59.873506 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-l5h6q" podStartSLOduration=1.183670271 podStartE2EDuration="21.873479312s" totalImagesPullingTime="20.689809041s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:07:59.873336015 +0000 UTC m=+22.676151468" watchObservedRunningTime="2026-07-02 19:07:59.873479312 +0000 UTC m=+22.676294741" Jul 02 19:07:59.887316 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:59.887145 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-161.ec2.internal" podStartSLOduration=20.887126954 podStartE2EDuration="20.887126954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:07:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:07:59.886488153 +0000 UTC m=+22.689303622" watchObservedRunningTime="2026-07-02 19:07:59.887126954 +0000 UTC m=+22.689942406" Jul 02 19:07:59.915285 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:07:59.912505 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vtngm" podStartSLOduration=1.4574956239999999 podStartE2EDuration="21.91245309s" totalImagesPullingTime="20.454957466s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:07:59.912082842 +0000 UTC m=+22.714898322" watchObservedRunningTime="2026-07-02 19:07:59.91245309 +0000 UTC m=+22.715268541" Jul 02 19:08:00.721462 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.721414 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:08:00.721586 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:00.721512 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:08:00.862237 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.862209 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vqm46" event={"ID":"798f8da0-e51d-47d2-b055-6456e8e9c1e1","Type":"ContainerStarted","Data":"c4a5e263dcdac332a16187dc85d47969704fb5fff51075a3042cbe69f88aebb6"} Jul 02 19:08:00.863564 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.863534 2578 generic.go:356] "Generic (PLEG): container finished" podID="05f91772ec7f4ad16d2e21378d51cdee" containerID="ef28b1bfc2d3a0bf96b2504db03c92e5e29d5476ac594f7ddab090bd34d459ca" exitCode=0 Jul 02 19:08:00.863648 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.863610 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal" event={"ID":"05f91772ec7f4ad16d2e21378d51cdee","Type":"ContainerDied","Data":"ef28b1bfc2d3a0bf96b2504db03c92e5e29d5476ac594f7ddab090bd34d459ca"} Jul 02 19:08:00.866451 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.866430 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" event={"ID":"9a0d9114-0c59-4c0d-8615-f98267f398b6","Type":"ContainerStarted","Data":"02d071e046f13196ed2cd79838c9bab08145d0cb9c793ece3c8f86f7d5dd36ff"} Jul 02 19:08:00.866552 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.866485 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" event={"ID":"9a0d9114-0c59-4c0d-8615-f98267f398b6","Type":"ContainerStarted","Data":"3d1c98bee08303158c73fece451641c749fb44492e04013c340a0dbb1b29278b"} Jul 02 19:08:00.866552 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.866524 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" event={"ID":"9a0d9114-0c59-4c0d-8615-f98267f398b6","Type":"ContainerStarted","Data":"abe80f0c25395a1d482a99b188a79dfa974399ae65932eb5373de53288d94dea"} Jul 02 19:08:00.866647 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.866562 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" event={"ID":"9a0d9114-0c59-4c0d-8615-f98267f398b6","Type":"ContainerStarted","Data":"58d1df1baeed1781399efc9f7b99f1d5777733b8d1a1f68a2f222aec5f7e194a"} Jul 02 19:08:00.866647 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.866593 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" event={"ID":"9a0d9114-0c59-4c0d-8615-f98267f398b6","Type":"ContainerStarted","Data":"9cce820806cdf1e5362fc2b8cd439065b4cff4d124e2c44dbe5231afed0130ed"} Jul 02 19:08:00.867658 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.867633 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ncfg2" event={"ID":"3b53746b-995c-49ce-a305-0993bd5f666b","Type":"ContainerStarted","Data":"0d025dfc9cc2beac7689765045e5f7c77c192eb84e7a27defa6f8588782207c2"} Jul 02 19:08:00.868850 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.868821 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-57hcx" event={"ID":"c0aec0a3-2014-4d88-a168-33b4b7048bb0","Type":"ContainerStarted","Data":"7b67eb01c325e98646955b69b2cb4c2bb76e0aa4f2cc94c4b3f628f100d22889"} Jul 02 19:08:00.869998 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.869979 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" event={"ID":"fe2caa8f-210e-4e84-89bd-f4158314d8c8","Type":"ContainerStarted","Data":"c7a31d14914e494dfec803299fe7d861fff73b7d898cad71e412744f70e8027e"} Jul 02 19:08:00.871267 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.871249 2578 generic.go:356] "Generic (PLEG): container finished" podID="365022fe-6d5e-4e74-be92-b791e81692b2" containerID="ed760734922fb942b26aa4145a0b50b37b843f4d6ef4cb553d62864816dce01b" exitCode=0 Jul 02 19:08:00.871342 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.871326 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prfg6" event={"ID":"365022fe-6d5e-4e74-be92-b791e81692b2","Type":"ContainerDied","Data":"ed760734922fb942b26aa4145a0b50b37b843f4d6ef4cb553d62864816dce01b"} Jul 02 19:08:00.872800 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.872665 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jkjv7" event={"ID":"8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc","Type":"ContainerStarted","Data":"5bfb09b316d311157e7339ec65e01a40be58d58511c6d90fadfd07ba7a23433b"} Jul 02 19:08:00.900914 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.900865 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vqm46" podStartSLOduration=2.207677324 podStartE2EDuration="22.900848416s" totalImagesPullingTime="20.693171092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:00.880772304 +0000 UTC m=+23.683587767" watchObservedRunningTime="2026-07-02 19:08:00.900848416 +0000 UTC m=+23.703663868" Jul 02 19:08:00.915111 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.915063 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-57hcx" podStartSLOduration=2.334658467 podStartE2EDuration="22.915046009s" totalImagesPullingTime="20.580387542s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:00.914597919 +0000 UTC m=+23.717413380" watchObservedRunningTime="2026-07-02 19:08:00.915046009 +0000 UTC m=+23.717861461" Jul 02 19:08:00.955341 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.955178 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ncfg2" podStartSLOduration=2.456318425 podStartE2EDuration="22.955165795s" totalImagesPullingTime="20.49884737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:00.95489302 +0000 UTC m=+23.757708471" watchObservedRunningTime="2026-07-02 19:08:00.955165795 +0000 UTC m=+23.757981246" Jul 02 19:08:00.971111 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:00.971068 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jkjv7" podStartSLOduration=2.275208567 podStartE2EDuration="22.97105449s" totalImagesPullingTime="20.695845923s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:00.970561088 +0000 UTC m=+23.773376540" watchObservedRunningTime="2026-07-02 19:08:00.97105449 +0000 UTC m=+23.773869941" Jul 02 19:08:01.722123 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:01.722097 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:08:01.722123 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:01.722113 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:08:01.722337 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:01.722217 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:08:01.722413 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:01.722344 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:08:01.826039 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:01.826015 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 19:08:01.877267 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:01.877233 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" event={"ID":"fe2caa8f-210e-4e84-89bd-f4158314d8c8","Type":"ContainerStarted","Data":"482a82b0ade69ac4ea618c0e4a9174b82d4b1f60f823c1380cf77e0634df0cd4"} Jul 02 19:08:01.879004 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:01.878979 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal" event={"ID":"05f91772ec7f4ad16d2e21378d51cdee","Type":"ContainerStarted","Data":"6ba0c7e2bd33e50c1a683645469c1dabfac835b8a2ea080fa79e738b02d85b30"} Jul 02 19:08:01.895077 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:01.895033 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-161.ec2.internal" podStartSLOduration=22.895020799 podStartE2EDuration="22.895020799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:07:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:01.894536838 +0000 UTC m=+24.697352288" watchObservedRunningTime="2026-07-02 19:08:01.895020799 +0000 UTC m=+24.697836250" Jul 02 19:08:02.525894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:02.525865 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vqm46" Jul 02 19:08:02.526426 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:02.526377 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vqm46" Jul 02 19:08:02.701622 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:02.701469 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T19:08:01.82603535Z","UUID":"b284da3b-4fe5-4667-a124-802ef22e6f16","Handler":null,"Name":"","Endpoint":""} Jul 02 19:08:02.705269 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:02.705245 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 19:08:02.705441 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:02.705306 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 19:08:02.720791 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:02.720767 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:08:02.720920 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:02.720878 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:08:02.883823 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:02.883789 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" event={"ID":"9a0d9114-0c59-4c0d-8615-f98267f398b6","Type":"ContainerStarted","Data":"b99bf4aec23ecc976b6c91aaf287445ddd3dfd0524abac4499173cd8ae013734"} Jul 02 19:08:02.884215 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:02.884066 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vqm46" Jul 02 19:08:02.884425 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:02.884390 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vqm46" Jul 02 19:08:03.721128 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:03.721082 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:08:03.721473 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:03.721221 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:08:03.721473 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:03.721271 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:08:03.721473 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:03.721358 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:08:03.887275 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:03.887242 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" event={"ID":"fe2caa8f-210e-4e84-89bd-f4158314d8c8","Type":"ContainerStarted","Data":"27d60d4e34f926b9030bc100b91d0c9a1a0de06f1537c8e3d013e0c8b5835dd9"} Jul 02 19:08:03.905645 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:03.905602 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jn7jc" podStartSLOduration=2.964725714 podStartE2EDuration="25.905588399s" totalImagesPullingTime="22.940862685s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:07:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:03.905282535 +0000 UTC m=+26.708097988" watchObservedRunningTime="2026-07-02 19:08:03.905588399 +0000 UTC m=+26.708403850" Jul 02 19:08:04.721177 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:04.721020 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:08:04.721314 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:04.721292 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:08:04.893235 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:04.893202 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" event={"ID":"9a0d9114-0c59-4c0d-8615-f98267f398b6","Type":"ContainerStarted","Data":"9f63a255f5e2ea75aa4658e74caa76e2b021d416176a7170dc072796cc2aec2f"} Jul 02 19:08:04.920895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:04.920835 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" podStartSLOduration=6.351172055 podStartE2EDuration="26.92081372s" totalImagesPullingTime="20.569641665s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:04.919379184 +0000 UTC m=+27.722194643" watchObservedRunningTime="2026-07-02 19:08:04.92081372 +0000 UTC m=+27.723629171" Jul 02 19:08:05.721369 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:05.721338 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:08:05.721564 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:05.721449 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:08:05.721564 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:05.721532 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:08:05.721676 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:05.721618 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:08:05.899825 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:05.899595 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:08:05.899825 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:05.899705 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:08:05.899825 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:05.899751 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:08:05.920459 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:05.920341 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:08:05.922840 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:05.922815 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:08:06.721619 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:06.721589 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:08:06.721780 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:06.721733 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:08:06.814561 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:06.814529 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9qvrm"] Jul 02 19:08:06.814723 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:06.814661 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:08:06.814793 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:06.814767 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:08:06.817743 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:06.817716 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p2dbb"] Jul 02 19:08:06.817870 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:06.817830 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:08:06.817963 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:06.817929 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:08:06.818346 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:06.818279 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rjlz4"] Jul 02 19:08:06.900831 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:06.900802 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:08:06.901437 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:06.900922 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:08:08.720907 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:08.720876 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:08:08.721299 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:08.720911 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:08:08.721299 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:08.720876 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:08:08.721299 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:08.720987 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:08:08.721299 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:08.721058 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:08:08.721299 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:08.721162 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:08:08.905933 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:08.905860 2578 generic.go:356] "Generic (PLEG): container finished" podID="365022fe-6d5e-4e74-be92-b791e81692b2" containerID="107c96a514e6c427d174a03a42844bbcce35aedcbce3a1c59c601d4dfc496026" exitCode=0 Jul 02 19:08:08.905933 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:08.905915 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prfg6" event={"ID":"365022fe-6d5e-4e74-be92-b791e81692b2","Type":"ContainerDied","Data":"107c96a514e6c427d174a03a42844bbcce35aedcbce3a1c59c601d4dfc496026"} Jul 02 19:08:09.910355 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:09.910271 2578 generic.go:356] "Generic (PLEG): container finished" podID="365022fe-6d5e-4e74-be92-b791e81692b2" containerID="cab92c0bababf1d18856183216f14181d8be7b6c8a9fbb4c575e39d594e56906" exitCode=0 Jul 02 19:08:09.910355 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:09.910329 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prfg6" event={"ID":"365022fe-6d5e-4e74-be92-b791e81692b2","Type":"ContainerDied","Data":"cab92c0bababf1d18856183216f14181d8be7b6c8a9fbb4c575e39d594e56906"} Jul 02 19:08:10.384859 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:10.384819 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5562\" (UniqueName: \"kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562\") pod \"network-check-target-9qvrm\" (UID: \"f43fc6d3-00d7-4153-aea4-423b1d40e67b\") " pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:08:10.385043 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:10.384997 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:08:10.385043 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:10.385024 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:08:10.385043 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:10.385036 2578 projected.go:196] Error preparing data for projected volume kube-api-access-q5562 for pod openshift-network-diagnostics/network-check-target-9qvrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:08:10.385203 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:10.385112 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562 podName:f43fc6d3-00d7-4153-aea4-423b1d40e67b nodeName:}" failed. No retries permitted until 2026-07-02 19:08:42.385088644 +0000 UTC m=+65.187904077 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-q5562" (UniqueName: "kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562") pod "network-check-target-9qvrm" (UID: "f43fc6d3-00d7-4153-aea4-423b1d40e67b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:08:10.485990 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:10.485960 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs\") pod \"network-metrics-daemon-rjlz4\" (UID: \"04c1f8e7-1552-41f6-b1af-d4ba971e3f76\") " pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:08:10.486137 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:10.486005 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret\") pod \"global-pull-secret-syncer-p2dbb\" (UID: \"7bd39ca6-f010-4e22-b7b9-04bf2acece57\") " pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:08:10.486174 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:10.486142 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:08:10.486174 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:10.486165 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:08:10.486253 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:10.486217 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs podName:04c1f8e7-1552-41f6-b1af-d4ba971e3f76 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:42.486202415 +0000 UTC m=+65.289017848 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs") pod "network-metrics-daemon-rjlz4" (UID: "04c1f8e7-1552-41f6-b1af-d4ba971e3f76") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:08:10.486253 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:10.486233 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret podName:7bd39ca6-f010-4e22-b7b9-04bf2acece57 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:42.486225899 +0000 UTC m=+65.289041332 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret") pod "global-pull-secret-syncer-p2dbb" (UID: "7bd39ca6-f010-4e22-b7b9-04bf2acece57") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:08:10.721562 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:10.721477 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:08:10.721708 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:10.721575 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:08:10.721708 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:10.721581 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:08:10.721708 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:10.721602 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:08:10.721708 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:10.721680 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:08:10.721839 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:10.721759 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:08:10.914927 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:10.914900 2578 generic.go:356] "Generic (PLEG): container finished" podID="365022fe-6d5e-4e74-be92-b791e81692b2" containerID="19161393ef67ffafb2e65a831513380952474ba86011641b41360c46bd88505e" exitCode=0 Jul 02 19:08:10.915288 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:10.914954 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prfg6" event={"ID":"365022fe-6d5e-4e74-be92-b791e81692b2","Type":"ContainerDied","Data":"19161393ef67ffafb2e65a831513380952474ba86011641b41360c46bd88505e"} Jul 02 19:08:12.721479 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:12.721448 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:08:12.721881 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:12.721570 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2dbb" podUID="7bd39ca6-f010-4e22-b7b9-04bf2acece57" Jul 02 19:08:12.721881 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:12.721448 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:08:12.721881 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:12.721448 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:08:12.721881 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:12.721670 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjlz4" podUID="04c1f8e7-1552-41f6-b1af-d4ba971e3f76" Jul 02 19:08:12.721881 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:12.721715 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qvrm" podUID="f43fc6d3-00d7-4153-aea4-423b1d40e67b" Jul 02 19:08:13.501371 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.501295 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-161.ec2.internal" event="NodeReady" Jul 02 19:08:13.501520 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.501457 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 19:08:13.546155 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.546125 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-xrtv2"] Jul 02 19:08:13.549895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.549875 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-74f4b96cdc-gnnfv"] Jul 02 19:08:13.550040 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.550011 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.552875 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.552850 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 19:08:13.553095 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.553073 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 02 19:08:13.553165 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.553131 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 19:08:13.553684 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.553476 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 02 19:08:13.553684 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.553584 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-shvf5\"" Jul 02 19:08:13.554927 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.554904 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f6886b98f-9sg6d"] Jul 02 19:08:13.555169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.555148 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:13.558709 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.558688 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 02 19:08:13.558830 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.558712 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 02 19:08:13.558933 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.558899 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q"] Jul 02 19:08:13.559074 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.559056 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.559443 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.559422 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 02 19:08:13.559775 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.559698 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 02 19:08:13.559775 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.559706 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 02 19:08:13.559920 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.559847 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-fjjwr\"" Jul 02 19:08:13.559971 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.559960 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 02 19:08:13.561537 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.561459 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 02 19:08:13.562766 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.562746 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" Jul 02 19:08:13.565337 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.565304 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 02 19:08:13.566173 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.566153 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 02 19:08:13.567944 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.567926 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 02 19:08:13.568150 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.568132 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-d4gw7\"" Jul 02 19:08:13.568230 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.568192 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 02 19:08:13.570252 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.570232 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 19:08:13.570332 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.570289 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 19:08:13.570332 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.570236 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 02 19:08:13.570460 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.570290 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 02 19:08:13.570460 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.570232 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-l2lvj\"" Jul 02 19:08:13.590917 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.590897 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q"] Jul 02 19:08:13.604624 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.604596 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f6886b98f-9sg6d"] Jul 02 19:08:13.608911 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.608890 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-xrtv2"] Jul 02 19:08:13.609623 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.609605 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74f4b96cdc-gnnfv"] Jul 02 19:08:13.683098 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.683057 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:13.707584 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.707558 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:13.707584 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.707588 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b744daf-9c04-4bb1-bfe7-babae8745b4a-ca-trust-extracted\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.707776 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.707615 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh64q\" (UniqueName: \"kubernetes.io/projected/9998d8a2-ea82-4b1f-aae9-554e79db6264-kube-api-access-zh64q\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:13.707776 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.707640 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b744daf-9c04-4bb1-bfe7-babae8745b4a-image-registry-private-configuration\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.707776 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.707656 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b744daf-9c04-4bb1-bfe7-babae8745b4a-installation-pull-secrets\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.707776 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.707683 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq6w4\" (UniqueName: \"kubernetes.io/projected/c8db16c6-0045-462c-a896-b66e91f4beb1-kube-api-access-bq6w4\") pod \"cluster-monitoring-operator-64786d9cbb-mlg8q\" (UID: \"c8db16c6-0045-462c-a896-b66e91f4beb1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" Jul 02 19:08:13.707776 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.707770 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.708029 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.707801 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b744daf-9c04-4bb1-bfe7-babae8745b4a-trusted-ca\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.708029 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.707833 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c8db16c6-0045-462c-a896-b66e91f4beb1-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-mlg8q\" (UID: \"c8db16c6-0045-462c-a896-b66e91f4beb1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" Jul 02 19:08:13.708029 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.707871 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-default-certificate\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:13.708029 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.707897 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-stats-auth\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:13.708029 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.707932 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bb75416-de32-44da-b7ae-29678f1cc099-tmp\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.708029 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.707965 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2bb75416-de32-44da-b7ae-29678f1cc099-serving-cert\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.708303 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.708050 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:13.708303 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.708082 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-bound-sa-token\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.708303 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.708096 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bb75416-de32-44da-b7ae-29678f1cc099-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.708303 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.708127 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bb75416-de32-44da-b7ae-29678f1cc099-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.708303 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.708161 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvhgh\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-kube-api-access-vvhgh\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.708303 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.708203 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drlnx\" (UniqueName: \"kubernetes.io/projected/2bb75416-de32-44da-b7ae-29678f1cc099-kube-api-access-drlnx\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.708303 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.708240 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mlg8q\" (UID: \"c8db16c6-0045-462c-a896-b66e91f4beb1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" Jul 02 19:08:13.708303 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.708279 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-certificates\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.708684 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.708307 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2bb75416-de32-44da-b7ae-29678f1cc099-snapshots\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.808761 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.808717 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2bb75416-de32-44da-b7ae-29678f1cc099-snapshots\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.809209 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.808896 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:13.809209 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.808933 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b744daf-9c04-4bb1-bfe7-babae8745b4a-ca-trust-extracted\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.809209 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.808960 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh64q\" (UniqueName: \"kubernetes.io/projected/9998d8a2-ea82-4b1f-aae9-554e79db6264-kube-api-access-zh64q\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:13.809209 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.808986 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b744daf-9c04-4bb1-bfe7-babae8745b4a-image-registry-private-configuration\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.809209 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:13.809052 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 19:08:13.809209 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.809156 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b744daf-9c04-4bb1-bfe7-babae8745b4a-installation-pull-secrets\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.809209 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.809196 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq6w4\" (UniqueName: \"kubernetes.io/projected/c8db16c6-0045-462c-a896-b66e91f4beb1-kube-api-access-bq6w4\") pod \"cluster-monitoring-operator-64786d9cbb-mlg8q\" (UID: \"c8db16c6-0045-462c-a896-b66e91f4beb1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" Jul 02 19:08:13.809525 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.809234 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.809525 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.809260 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b744daf-9c04-4bb1-bfe7-babae8745b4a-trusted-ca\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.809525 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.809289 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c8db16c6-0045-462c-a896-b66e91f4beb1-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-mlg8q\" (UID: \"c8db16c6-0045-462c-a896-b66e91f4beb1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" Jul 02 19:08:13.809525 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.809316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-default-certificate\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:13.809525 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.809340 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-stats-auth\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:13.809525 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.809357 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b744daf-9c04-4bb1-bfe7-babae8745b4a-ca-trust-extracted\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.809525 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:13.809386 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs podName:9998d8a2-ea82-4b1f-aae9-554e79db6264 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:14.30935799 +0000 UTC m=+37.112173423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs") pod "router-default-74f4b96cdc-gnnfv" (UID: "9998d8a2-ea82-4b1f-aae9-554e79db6264") : secret "router-metrics-certs-default" not found Jul 02 19:08:13.809525 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.809433 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bb75416-de32-44da-b7ae-29678f1cc099-tmp\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.809525 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.809477 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2bb75416-de32-44da-b7ae-29678f1cc099-serving-cert\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.809525 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.809499 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2bb75416-de32-44da-b7ae-29678f1cc099-snapshots\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.809525 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.809519 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:13.809984 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:13.809639 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle podName:9998d8a2-ea82-4b1f-aae9-554e79db6264 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:14.309625072 +0000 UTC m=+37.112440522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle") pod "router-default-74f4b96cdc-gnnfv" (UID: "9998d8a2-ea82-4b1f-aae9-554e79db6264") : configmap references non-existent config key: service-ca.crt Jul 02 19:08:13.809984 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.809906 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bb75416-de32-44da-b7ae-29678f1cc099-tmp\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.810212 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.810186 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c8db16c6-0045-462c-a896-b66e91f4beb1-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-mlg8q\" (UID: \"c8db16c6-0045-462c-a896-b66e91f4beb1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" Jul 02 19:08:13.810272 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:13.810245 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 19:08:13.810272 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:13.810259 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f6886b98f-9sg6d: secret "image-registry-tls" not found Jul 02 19:08:13.810383 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:13.810310 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls podName:8b744daf-9c04-4bb1-bfe7-babae8745b4a nodeName:}" failed. No retries permitted until 2026-07-02 19:08:14.310297478 +0000 UTC m=+37.113112924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls") pod "image-registry-7f6886b98f-9sg6d" (UID: "8b744daf-9c04-4bb1-bfe7-babae8745b4a") : secret "image-registry-tls" not found Jul 02 19:08:13.810383 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.810340 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-bound-sa-token\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.810383 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.810364 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bb75416-de32-44da-b7ae-29678f1cc099-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.810522 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.810422 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bb75416-de32-44da-b7ae-29678f1cc099-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.810522 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.810445 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvhgh\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-kube-api-access-vvhgh\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.810522 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.810450 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b744daf-9c04-4bb1-bfe7-babae8745b4a-trusted-ca\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.810522 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.810467 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drlnx\" (UniqueName: \"kubernetes.io/projected/2bb75416-de32-44da-b7ae-29678f1cc099-kube-api-access-drlnx\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.810522 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.810494 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mlg8q\" (UID: \"c8db16c6-0045-462c-a896-b66e91f4beb1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" Jul 02 19:08:13.810522 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.810520 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-certificates\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.811191 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.811146 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-certificates\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.811293 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:13.811232 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 19:08:13.811293 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:13.811273 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls podName:c8db16c6-0045-462c-a896-b66e91f4beb1 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:14.311260119 +0000 UTC m=+37.114075552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-mlg8q" (UID: "c8db16c6-0045-462c-a896-b66e91f4beb1") : secret "cluster-monitoring-operator-tls" not found Jul 02 19:08:13.811293 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.811231 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bb75416-de32-44da-b7ae-29678f1cc099-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.811750 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.811725 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bb75416-de32-44da-b7ae-29678f1cc099-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.814026 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.814000 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b744daf-9c04-4bb1-bfe7-babae8745b4a-image-registry-private-configuration\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.814112 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.814005 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2bb75416-de32-44da-b7ae-29678f1cc099-serving-cert\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.814112 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.814035 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b744daf-9c04-4bb1-bfe7-babae8745b4a-installation-pull-secrets\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.814112 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.814100 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-stats-auth\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:13.814242 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.814115 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-default-certificate\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:13.828721 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.828698 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh64q\" (UniqueName: \"kubernetes.io/projected/9998d8a2-ea82-4b1f-aae9-554e79db6264-kube-api-access-zh64q\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:13.830825 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.830804 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvhgh\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-kube-api-access-vvhgh\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.831485 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.831462 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drlnx\" (UniqueName: \"kubernetes.io/projected/2bb75416-de32-44da-b7ae-29678f1cc099-kube-api-access-drlnx\") pod \"insights-operator-5c847dcfc4-xrtv2\" (UID: \"2bb75416-de32-44da-b7ae-29678f1cc099\") " pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.831985 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.831962 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq6w4\" (UniqueName: \"kubernetes.io/projected/c8db16c6-0045-462c-a896-b66e91f4beb1-kube-api-access-bq6w4\") pod \"cluster-monitoring-operator-64786d9cbb-mlg8q\" (UID: \"c8db16c6-0045-462c-a896-b66e91f4beb1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" Jul 02 19:08:13.833355 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.833333 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-bound-sa-token\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:13.844802 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.844780 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-24ghv"] Jul 02 19:08:13.849759 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.849742 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:13.850730 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.850695 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:13.852523 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.852503 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 19:08:13.852599 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.852530 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 19:08:13.852599 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.852558 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-45sjn\"" Jul 02 19:08:13.860314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.860292 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-24ghv"] Jul 02 19:08:13.866308 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.866291 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" Jul 02 19:08:13.875798 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.875778 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-42jp6"] Jul 02 19:08:13.879065 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.879049 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-42jp6" Jul 02 19:08:13.882172 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.882155 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pjv9d\"" Jul 02 19:08:13.882253 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.882232 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 19:08:13.882321 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.882300 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 19:08:13.882456 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.882415 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 19:08:13.890032 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.890012 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-42jp6"] Jul 02 19:08:13.950657 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:13.950621 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:14.012712 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.012675 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9340dfab-6698-4817-82fa-e976787cfebb-cert\") pod \"ingress-canary-42jp6\" (UID: \"9340dfab-6698-4817-82fa-e976787cfebb\") " pod="openshift-ingress-canary/ingress-canary-42jp6" Jul 02 19:08:14.012855 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.012770 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wqls\" (UniqueName: \"kubernetes.io/projected/9340dfab-6698-4817-82fa-e976787cfebb-kube-api-access-5wqls\") pod \"ingress-canary-42jp6\" (UID: \"9340dfab-6698-4817-82fa-e976787cfebb\") " pod="openshift-ingress-canary/ingress-canary-42jp6" Jul 02 19:08:14.012855 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.012799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krds7\" (UniqueName: \"kubernetes.io/projected/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-kube-api-access-krds7\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:14.012935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.012911 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:14.012970 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.012937 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-tmp-dir\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:14.013007 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.012986 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-config-volume\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:14.051621 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.051587 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:14.113652 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.113559 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wqls\" (UniqueName: \"kubernetes.io/projected/9340dfab-6698-4817-82fa-e976787cfebb-kube-api-access-5wqls\") pod \"ingress-canary-42jp6\" (UID: \"9340dfab-6698-4817-82fa-e976787cfebb\") " pod="openshift-ingress-canary/ingress-canary-42jp6" Jul 02 19:08:14.113652 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.113604 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krds7\" (UniqueName: \"kubernetes.io/projected/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-kube-api-access-krds7\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:14.113916 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.113683 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:14.113916 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.113705 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-tmp-dir\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:14.113916 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.113734 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-config-volume\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:14.113916 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:14.113913 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:08:14.114095 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:14.113997 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls podName:b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:14.613975652 +0000 UTC m=+37.416791099 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls") pod "dns-default-24ghv" (UID: "b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71") : secret "dns-default-metrics-tls" not found Jul 02 19:08:14.114095 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.114059 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9340dfab-6698-4817-82fa-e976787cfebb-cert\") pod \"ingress-canary-42jp6\" (UID: \"9340dfab-6698-4817-82fa-e976787cfebb\") " pod="openshift-ingress-canary/ingress-canary-42jp6" Jul 02 19:08:14.114192 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.114120 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-tmp-dir\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:14.114252 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.114234 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-config-volume\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:14.116797 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.116774 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9340dfab-6698-4817-82fa-e976787cfebb-cert\") pod \"ingress-canary-42jp6\" (UID: \"9340dfab-6698-4817-82fa-e976787cfebb\") " pod="openshift-ingress-canary/ingress-canary-42jp6" Jul 02 19:08:14.123012 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.122993 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krds7\" (UniqueName: \"kubernetes.io/projected/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-kube-api-access-krds7\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:14.123304 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.123286 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wqls\" (UniqueName: \"kubernetes.io/projected/9340dfab-6698-4817-82fa-e976787cfebb-kube-api-access-5wqls\") pod \"ingress-canary-42jp6\" (UID: \"9340dfab-6698-4817-82fa-e976787cfebb\") " pod="openshift-ingress-canary/ingress-canary-42jp6" Jul 02 19:08:14.189201 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.189174 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-42jp6" Jul 02 19:08:14.316055 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.316012 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:14.316271 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.316076 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mlg8q\" (UID: \"c8db16c6-0045-462c-a896-b66e91f4beb1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" Jul 02 19:08:14.316271 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.316127 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:14.316271 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.316169 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:14.316271 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:14.316196 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle podName:9998d8a2-ea82-4b1f-aae9-554e79db6264 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:15.316174987 +0000 UTC m=+38.118990431 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle") pod "router-default-74f4b96cdc-gnnfv" (UID: "9998d8a2-ea82-4b1f-aae9-554e79db6264") : configmap references non-existent config key: service-ca.crt Jul 02 19:08:14.316271 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:14.316258 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 19:08:14.316527 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:14.316308 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 19:08:14.316527 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:14.316263 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 19:08:14.316527 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:14.316327 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls podName:c8db16c6-0045-462c-a896-b66e91f4beb1 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:15.316310767 +0000 UTC m=+38.119126200 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-mlg8q" (UID: "c8db16c6-0045-462c-a896-b66e91f4beb1") : secret "cluster-monitoring-operator-tls" not found Jul 02 19:08:14.316527 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:14.316335 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f6886b98f-9sg6d: secret "image-registry-tls" not found Jul 02 19:08:14.316527 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:14.316352 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs podName:9998d8a2-ea82-4b1f-aae9-554e79db6264 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:15.316338383 +0000 UTC m=+38.119153815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs") pod "router-default-74f4b96cdc-gnnfv" (UID: "9998d8a2-ea82-4b1f-aae9-554e79db6264") : secret "router-metrics-certs-default" not found Jul 02 19:08:14.316527 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:14.316373 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls podName:8b744daf-9c04-4bb1-bfe7-babae8745b4a nodeName:}" failed. No retries permitted until 2026-07-02 19:08:15.316364004 +0000 UTC m=+38.119179433 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls") pod "image-registry-7f6886b98f-9sg6d" (UID: "8b744daf-9c04-4bb1-bfe7-babae8745b4a") : secret "image-registry-tls" not found Jul 02 19:08:14.365482 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.365342 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:14.466312 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.466204 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:14.541986 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.541930 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:14.619468 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.619374 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:14.619642 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:14.619510 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:08:14.619642 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:14.619584 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls podName:b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:15.619562817 +0000 UTC m=+38.422378250 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls") pod "dns-default-24ghv" (UID: "b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71") : secret "dns-default-metrics-tls" not found Jul 02 19:08:14.721875 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.721843 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:08:14.721875 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.721862 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:08:14.722109 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.721843 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:08:14.725148 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.725124 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 19:08:14.726055 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.726035 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 19:08:14.726165 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.726074 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5rf5h\"" Jul 02 19:08:14.726165 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.726079 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 19:08:14.726165 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.726104 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-r7xgd\"" Jul 02 19:08:14.726322 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.726311 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 19:08:14.849132 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.849105 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:14.914147 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:14.914062 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:15.325193 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:15.325157 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:15.325332 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:15.325204 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mlg8q\" (UID: \"c8db16c6-0045-462c-a896-b66e91f4beb1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" Jul 02 19:08:15.325332 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:15.325247 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:15.325332 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:15.325276 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:15.325511 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:15.325344 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle podName:9998d8a2-ea82-4b1f-aae9-554e79db6264 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:17.325325871 +0000 UTC m=+40.128141306 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle") pod "router-default-74f4b96cdc-gnnfv" (UID: "9998d8a2-ea82-4b1f-aae9-554e79db6264") : configmap references non-existent config key: service-ca.crt Jul 02 19:08:15.325511 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:15.325391 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 19:08:15.325511 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:15.325430 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 19:08:15.325511 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:15.325456 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls podName:c8db16c6-0045-462c-a896-b66e91f4beb1 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:17.325445376 +0000 UTC m=+40.128260808 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-mlg8q" (UID: "c8db16c6-0045-462c-a896-b66e91f4beb1") : secret "cluster-monitoring-operator-tls" not found Jul 02 19:08:15.325511 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:15.325491 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs podName:9998d8a2-ea82-4b1f-aae9-554e79db6264 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:17.325472003 +0000 UTC m=+40.128287433 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs") pod "router-default-74f4b96cdc-gnnfv" (UID: "9998d8a2-ea82-4b1f-aae9-554e79db6264") : secret "router-metrics-certs-default" not found Jul 02 19:08:15.325511 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:15.325494 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 19:08:15.325511 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:15.325513 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f6886b98f-9sg6d: secret "image-registry-tls" not found Jul 02 19:08:15.325735 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:15.325558 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls podName:8b744daf-9c04-4bb1-bfe7-babae8745b4a nodeName:}" failed. No retries permitted until 2026-07-02 19:08:17.32554506 +0000 UTC m=+40.128360503 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls") pod "image-registry-7f6886b98f-9sg6d" (UID: "8b744daf-9c04-4bb1-bfe7-babae8745b4a") : secret "image-registry-tls" not found Jul 02 19:08:15.336134 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:15.336112 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-xrtv2"] Jul 02 19:08:15.339800 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:15.339781 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-42jp6"] Jul 02 19:08:15.420904 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:08:15.420861 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bb75416_de32_44da_b7ae_29678f1cc099.slice/crio-369406c94f551bda1cd337fb732e11b7dfd9649d3e1ab9b731f930810fd62a00 WatchSource:0}: Error finding container 369406c94f551bda1cd337fb732e11b7dfd9649d3e1ab9b731f930810fd62a00: Status 404 returned error can't find the container with id 369406c94f551bda1cd337fb732e11b7dfd9649d3e1ab9b731f930810fd62a00 Jul 02 19:08:15.421131 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:08:15.421113 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9340dfab_6698_4817_82fa_e976787cfebb.slice/crio-f8efbb16c57c7235df71c0ce50bdb34b601efb2d58504d96d7a67c02588ab23a WatchSource:0}: Error finding container f8efbb16c57c7235df71c0ce50bdb34b601efb2d58504d96d7a67c02588ab23a: Status 404 returned error can't find the container with id f8efbb16c57c7235df71c0ce50bdb34b601efb2d58504d96d7a67c02588ab23a Jul 02 19:08:15.627811 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:15.627781 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:15.627974 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:15.627900 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:08:15.627974 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:15.627948 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls podName:b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:17.627936028 +0000 UTC m=+40.430751456 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls") pod "dns-default-24ghv" (UID: "b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71") : secret "dns-default-metrics-tls" not found Jul 02 19:08:15.631975 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:15.631946 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:15.732378 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:15.732293 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:15.928853 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:15.928820 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" event={"ID":"2bb75416-de32-44da-b7ae-29678f1cc099","Type":"ContainerStarted","Data":"369406c94f551bda1cd337fb732e11b7dfd9649d3e1ab9b731f930810fd62a00"} Jul 02 19:08:15.933192 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:15.933120 2578 generic.go:356] "Generic (PLEG): container finished" podID="365022fe-6d5e-4e74-be92-b791e81692b2" containerID="92751e9b083fb768aa7bb23aa36d2e8635ffa8832cd1ea00bee6786f08c87a97" exitCode=0 Jul 02 19:08:15.933336 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:15.933197 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prfg6" event={"ID":"365022fe-6d5e-4e74-be92-b791e81692b2","Type":"ContainerDied","Data":"92751e9b083fb768aa7bb23aa36d2e8635ffa8832cd1ea00bee6786f08c87a97"} Jul 02 19:08:15.935018 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:15.934993 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-42jp6" event={"ID":"9340dfab-6698-4817-82fa-e976787cfebb","Type":"ContainerStarted","Data":"f8efbb16c57c7235df71c0ce50bdb34b601efb2d58504d96d7a67c02588ab23a"} Jul 02 19:08:16.037453 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:16.037268 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:16.141036 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:16.140989 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:16.940782 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:16.940754 2578 generic.go:356] "Generic (PLEG): container finished" podID="365022fe-6d5e-4e74-be92-b791e81692b2" containerID="bfb387f6723916283a55323609d359ef5f4466a09155bd1331772b5bc8fb3277" exitCode=0 Jul 02 19:08:16.941146 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:16.940813 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prfg6" event={"ID":"365022fe-6d5e-4e74-be92-b791e81692b2","Type":"ContainerDied","Data":"bfb387f6723916283a55323609d359ef5f4466a09155bd1331772b5bc8fb3277"} Jul 02 19:08:17.296333 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:17.296227 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:17.342940 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:17.342909 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mlg8q\" (UID: \"c8db16c6-0045-462c-a896-b66e91f4beb1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" Jul 02 19:08:17.343153 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:17.342968 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:17.343153 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:17.343012 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:17.343153 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:17.343075 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 19:08:17.343153 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:17.343096 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:17.343153 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:17.343148 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls podName:c8db16c6-0045-462c-a896-b66e91f4beb1 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:21.343126857 +0000 UTC m=+44.145942299 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-mlg8q" (UID: "c8db16c6-0045-462c-a896-b66e91f4beb1") : secret "cluster-monitoring-operator-tls" not found Jul 02 19:08:17.343446 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:17.343083 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 19:08:17.343446 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:17.343166 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 19:08:17.343446 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:17.343180 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f6886b98f-9sg6d: secret "image-registry-tls" not found Jul 02 19:08:17.343446 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:17.343220 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs podName:9998d8a2-ea82-4b1f-aae9-554e79db6264 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:21.343203354 +0000 UTC m=+44.146018798 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs") pod "router-default-74f4b96cdc-gnnfv" (UID: "9998d8a2-ea82-4b1f-aae9-554e79db6264") : secret "router-metrics-certs-default" not found Jul 02 19:08:17.343446 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:17.343238 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle podName:9998d8a2-ea82-4b1f-aae9-554e79db6264 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:21.343228712 +0000 UTC m=+44.146044154 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle") pod "router-default-74f4b96cdc-gnnfv" (UID: "9998d8a2-ea82-4b1f-aae9-554e79db6264") : configmap references non-existent config key: service-ca.crt Jul 02 19:08:17.343446 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:17.343251 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls podName:8b744daf-9c04-4bb1-bfe7-babae8745b4a nodeName:}" failed. No retries permitted until 2026-07-02 19:08:21.34324384 +0000 UTC m=+44.146059284 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls") pod "image-registry-7f6886b98f-9sg6d" (UID: "8b744daf-9c04-4bb1-bfe7-babae8745b4a") : secret "image-registry-tls" not found Jul 02 19:08:17.645689 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:17.645642 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:17.645875 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:17.645801 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:08:17.645875 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:17.645863 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls podName:b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:21.645848004 +0000 UTC m=+44.448663446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls") pod "dns-default-24ghv" (UID: "b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71") : secret "dns-default-metrics-tls" not found Jul 02 19:08:17.946785 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:17.946706 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prfg6" event={"ID":"365022fe-6d5e-4e74-be92-b791e81692b2","Type":"ContainerStarted","Data":"31567a546c7180b289599597ffe99e1a35bceed354c497073d6768dae7327447"} Jul 02 19:08:17.968870 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:17.968820 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-prfg6" podStartSLOduration=6.443400581 podStartE2EDuration="39.96880142s" totalImagesPullingTime="33.525400839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:07:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:17.967939493 +0000 UTC m=+40.770754982" watchObservedRunningTime="2026-07-02 19:08:17.96880142 +0000 UTC m=+40.771616872" Jul 02 19:08:18.110237 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:18.110199 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:18.193692 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:18.193656 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:18.795865 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:18.795836 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:18.854737 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:18.854710 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:18.951999 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:18.951961 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-42jp6" event={"ID":"9340dfab-6698-4817-82fa-e976787cfebb","Type":"ContainerStarted","Data":"888d88a411967fd2b55e98153d7c42b8be1f9ab6573d788ba119f1738bd2dc11"} Jul 02 19:08:18.953559 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:18.953531 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" event={"ID":"2bb75416-de32-44da-b7ae-29678f1cc099","Type":"ContainerStarted","Data":"3a10270d089b3391cbb6f443aaa6f96c5789e93068b5650d1a5b012453f50f7a"} Jul 02 19:08:18.967567 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:18.967518 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-42jp6" podStartSLOduration=2.616213008 podStartE2EDuration="5.967500715s" totalImagesPullingTime="3.351287707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:18.966661069 +0000 UTC m=+41.769476530" watchObservedRunningTime="2026-07-02 19:08:18.967500715 +0000 UTC m=+41.770316168" Jul 02 19:08:18.984996 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:18.984945 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" podStartSLOduration=7.640517912 podStartE2EDuration="10.98492752s" totalImagesPullingTime="3.344409608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:18.984093658 +0000 UTC m=+41.786909133" watchObservedRunningTime="2026-07-02 19:08:18.98492752 +0000 UTC m=+41.787742973" Jul 02 19:08:19.866211 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:19.866171 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:19.967991 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:19.967966 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:20.266762 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.266687 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-p47cx"] Jul 02 19:08:20.298345 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.298317 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:20.301381 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.301332 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p47cx"] Jul 02 19:08:20.308238 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.308157 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 19:08:20.309142 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.309110 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jjnz9\"" Jul 02 19:08:20.309333 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.309316 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 19:08:20.316303 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.316208 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:20.367845 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.367822 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:20.367974 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.367867 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d12a1023-4e31-4eb4-9aab-0b737eb11f51-data-volume\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:20.367974 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.367921 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:20.367974 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.367961 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqpd4\" (UniqueName: \"kubernetes.io/projected/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-api-access-cqpd4\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:20.368092 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.367985 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d12a1023-4e31-4eb4-9aab-0b737eb11f51-crio-socket\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:20.388494 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.388462 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:20.468871 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.468846 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:20.468970 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.468886 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d12a1023-4e31-4eb4-9aab-0b737eb11f51-data-volume\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:20.468970 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.468913 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:20.469051 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.468965 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqpd4\" (UniqueName: \"kubernetes.io/projected/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-api-access-cqpd4\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:20.469051 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:20.468975 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 19:08:20.469051 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.468995 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d12a1023-4e31-4eb4-9aab-0b737eb11f51-crio-socket\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:20.469051 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:20.468999 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 19:08:20.469051 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:20.469045 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm podName:d12a1023-4e31-4eb4-9aab-0b737eb11f51 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:20.96902755 +0000 UTC m=+43.771842986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p47cx" (UID: "d12a1023-4e31-4eb4-9aab-0b737eb11f51") : configmap "kube-rbac-proxy" not found Jul 02 19:08:20.469272 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:20.469167 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls podName:d12a1023-4e31-4eb4-9aab-0b737eb11f51 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:20.969146617 +0000 UTC m=+43.771962054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p47cx" (UID: "d12a1023-4e31-4eb4-9aab-0b737eb11f51") : secret "insights-runtime-extractor-tls" not found Jul 02 19:08:20.469272 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.469211 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d12a1023-4e31-4eb4-9aab-0b737eb11f51-crio-socket\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:20.469272 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.469267 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d12a1023-4e31-4eb4-9aab-0b737eb11f51-data-volume\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:20.479094 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.479069 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqpd4\" (UniqueName: \"kubernetes.io/projected/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-api-access-cqpd4\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:20.773826 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.773793 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:20.875804 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.875772 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:20.972595 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.972567 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:20.972925 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:20.972609 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:20.972925 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:20.972680 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 19:08:20.972925 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:20.972700 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 19:08:20.972925 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:20.972743 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls podName:d12a1023-4e31-4eb4-9aab-0b737eb11f51 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:21.972729717 +0000 UTC m=+44.775545146 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p47cx" (UID: "d12a1023-4e31-4eb4-9aab-0b737eb11f51") : secret "insights-runtime-extractor-tls" not found Jul 02 19:08:20.972925 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:20.972798 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm podName:d12a1023-4e31-4eb4-9aab-0b737eb11f51 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:21.972792492 +0000 UTC m=+44.775607920 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p47cx" (UID: "d12a1023-4e31-4eb4-9aab-0b737eb11f51") : configmap "kube-rbac-proxy" not found Jul 02 19:08:21.375662 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:21.375626 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:21.375856 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:21.375673 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mlg8q\" (UID: \"c8db16c6-0045-462c-a896-b66e91f4beb1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" Jul 02 19:08:21.375856 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:21.375719 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:21.375856 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:21.375759 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:21.375856 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:21.375823 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle podName:9998d8a2-ea82-4b1f-aae9-554e79db6264 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:29.375805397 +0000 UTC m=+52.178620826 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle") pod "router-default-74f4b96cdc-gnnfv" (UID: "9998d8a2-ea82-4b1f-aae9-554e79db6264") : configmap references non-existent config key: service-ca.crt Jul 02 19:08:21.375856 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:21.375832 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 19:08:21.376077 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:21.375836 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 19:08:21.376077 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:21.375872 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 19:08:21.376077 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:21.375880 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls podName:c8db16c6-0045-462c-a896-b66e91f4beb1 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:29.375868413 +0000 UTC m=+52.178683842 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-mlg8q" (UID: "c8db16c6-0045-462c-a896-b66e91f4beb1") : secret "cluster-monitoring-operator-tls" not found Jul 02 19:08:21.376077 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:21.375887 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f6886b98f-9sg6d: secret "image-registry-tls" not found Jul 02 19:08:21.376077 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:21.375904 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs podName:9998d8a2-ea82-4b1f-aae9-554e79db6264 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:29.375895281 +0000 UTC m=+52.178710715 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs") pod "router-default-74f4b96cdc-gnnfv" (UID: "9998d8a2-ea82-4b1f-aae9-554e79db6264") : secret "router-metrics-certs-default" not found Jul 02 19:08:21.376077 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:21.375930 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls podName:8b744daf-9c04-4bb1-bfe7-babae8745b4a nodeName:}" failed. No retries permitted until 2026-07-02 19:08:29.375913978 +0000 UTC m=+52.178729419 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls") pod "image-registry-7f6886b98f-9sg6d" (UID: "8b744daf-9c04-4bb1-bfe7-babae8745b4a") : secret "image-registry-tls" not found Jul 02 19:08:21.679354 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:21.679272 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:21.679519 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:21.679429 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:08:21.679519 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:21.679489 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls podName:b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:29.679474614 +0000 UTC m=+52.482290043 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls") pod "dns-default-24ghv" (UID: "b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71") : secret "dns-default-metrics-tls" not found Jul 02 19:08:21.982587 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:21.982518 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:21.982587 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:21.982561 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:21.982982 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:21.982631 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 19:08:21.982982 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:21.982714 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm podName:d12a1023-4e31-4eb4-9aab-0b737eb11f51 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:23.982683725 +0000 UTC m=+46.785499154 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p47cx" (UID: "d12a1023-4e31-4eb4-9aab-0b737eb11f51") : configmap "kube-rbac-proxy" not found Jul 02 19:08:21.982982 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:21.982738 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 19:08:21.982982 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:21.982793 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls podName:d12a1023-4e31-4eb4-9aab-0b737eb11f51 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:23.982779008 +0000 UTC m=+46.785594451 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p47cx" (UID: "d12a1023-4e31-4eb4-9aab-0b737eb11f51") : secret "insights-runtime-extractor-tls" not found Jul 02 19:08:22.020926 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:22.020903 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jkjv7_8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc/dns-node-resolver/0.log" Jul 02 19:08:22.420475 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:22.420450 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-57hcx_c0aec0a3-2014-4d88-a168-33b4b7048bb0/node-ca/0.log" Jul 02 19:08:23.220941 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:23.220894 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-42jp6_9340dfab-6698-4817-82fa-e976787cfebb/serve-healthcheck-canary/0.log" Jul 02 19:08:23.999860 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:23.999829 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:24.000026 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:23.999945 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:24.000026 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:23.999971 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 19:08:24.000026 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:24.000018 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 19:08:24.000126 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:24.000030 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls podName:d12a1023-4e31-4eb4-9aab-0b737eb11f51 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:28.000014858 +0000 UTC m=+50.802830307 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p47cx" (UID: "d12a1023-4e31-4eb4-9aab-0b737eb11f51") : secret "insights-runtime-extractor-tls" not found Jul 02 19:08:24.000126 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:24.000056 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm podName:d12a1023-4e31-4eb4-9aab-0b737eb11f51 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:28.000045195 +0000 UTC m=+50.802860644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p47cx" (UID: "d12a1023-4e31-4eb4-9aab-0b737eb11f51") : configmap "kube-rbac-proxy" not found Jul 02 19:08:24.624737 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:24.624702 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:24.684923 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:24.684892 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:25.054504 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.054426 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:25.157443 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.157388 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:25.259806 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.259774 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:25.343084 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.343055 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-5kzdc"] Jul 02 19:08:25.347368 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.347340 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:25.384468 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.384442 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-5kzdc"] Jul 02 19:08:25.384571 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.384545 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5kzdc" Jul 02 19:08:25.386857 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.386838 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 02 19:08:25.386970 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.386954 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 02 19:08:25.387097 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.387083 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-bqtvk\"" Jul 02 19:08:25.406431 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.406411 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:25.411598 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.411581 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws76c\" (UniqueName: \"kubernetes.io/projected/317197ef-bfcd-4aca-9484-95d9b7d15183-kube-api-access-ws76c\") pod \"migrator-d57d58cc9-5kzdc\" (UID: \"317197ef-bfcd-4aca-9484-95d9b7d15183\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5kzdc" Jul 02 19:08:25.506130 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.506106 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:25.512105 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.512082 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws76c\" (UniqueName: \"kubernetes.io/projected/317197ef-bfcd-4aca-9484-95d9b7d15183-kube-api-access-ws76c\") pod \"migrator-d57d58cc9-5kzdc\" (UID: \"317197ef-bfcd-4aca-9484-95d9b7d15183\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5kzdc" Jul 02 19:08:25.520752 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.520731 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws76c\" (UniqueName: \"kubernetes.io/projected/317197ef-bfcd-4aca-9484-95d9b7d15183-kube-api-access-ws76c\") pod \"migrator-d57d58cc9-5kzdc\" (UID: \"317197ef-bfcd-4aca-9484-95d9b7d15183\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5kzdc" Jul 02 19:08:25.693282 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.693208 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5kzdc" Jul 02 19:08:25.838210 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.838181 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-5kzdc"] Jul 02 19:08:25.841131 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:08:25.841105 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod317197ef_bfcd_4aca_9484_95d9b7d15183.slice/crio-3494b4bf349525e7db7d042340761f55fa93a8456c712bc3d7b82e40f5aba234 WatchSource:0}: Error finding container 3494b4bf349525e7db7d042340761f55fa93a8456c712bc3d7b82e40f5aba234: Status 404 returned error can't find the container with id 3494b4bf349525e7db7d042340761f55fa93a8456c712bc3d7b82e40f5aba234 Jul 02 19:08:25.971007 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:25.970927 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5kzdc" event={"ID":"317197ef-bfcd-4aca-9484-95d9b7d15183","Type":"ContainerStarted","Data":"3494b4bf349525e7db7d042340761f55fa93a8456c712bc3d7b82e40f5aba234"} Jul 02 19:08:26.990787 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:26.990755 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:27.062015 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:27.061979 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:27.901961 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:27.901931 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:27.977241 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:27.977211 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5kzdc" event={"ID":"317197ef-bfcd-4aca-9484-95d9b7d15183","Type":"ContainerStarted","Data":"23a1c43b37a0af36f51836e8adf1543f0b41ef8f8832282c0c8e16a1b6c00f35"} Jul 02 19:08:28.031552 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:28.031529 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:28.031835 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:28.031572 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:28.031835 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:28.031643 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 19:08:28.031835 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:28.031688 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 19:08:28.031835 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:28.031699 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm podName:d12a1023-4e31-4eb4-9aab-0b737eb11f51 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:36.031681717 +0000 UTC m=+58.834497161 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p47cx" (UID: "d12a1023-4e31-4eb4-9aab-0b737eb11f51") : configmap "kube-rbac-proxy" not found Jul 02 19:08:28.031835 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:28.031723 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls podName:d12a1023-4e31-4eb4-9aab-0b737eb11f51 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:36.031713563 +0000 UTC m=+58.834529002 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p47cx" (UID: "d12a1023-4e31-4eb4-9aab-0b737eb11f51") : secret "insights-runtime-extractor-tls" not found Jul 02 19:08:28.512625 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:28.512593 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:28.981291 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:28.981259 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5kzdc" event={"ID":"317197ef-bfcd-4aca-9484-95d9b7d15183","Type":"ContainerStarted","Data":"ca9127fdba2764a921a4d0f5035db18cb1f5ee008c60661ec423e4cd9f97a63c"} Jul 02 19:08:29.003614 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:29.003568 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5kzdc" podStartSLOduration=2.054554239 podStartE2EDuration="4.003554226s" totalImagesPullingTime="1.948999987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:29.00270172 +0000 UTC m=+51.805517172" watchObservedRunningTime="2026-07-02 19:08:29.003554226 +0000 UTC m=+51.806369678" Jul 02 19:08:29.364083 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:29.364038 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:29.421290 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:29.421254 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:29.443651 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:29.443622 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:29.443787 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:29.443665 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mlg8q\" (UID: \"c8db16c6-0045-462c-a896-b66e91f4beb1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" Jul 02 19:08:29.443787 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:29.443715 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:29.443787 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:29.443757 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:29.443787 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:29.443781 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle podName:9998d8a2-ea82-4b1f-aae9-554e79db6264 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:45.443763614 +0000 UTC m=+68.246579042 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle") pod "router-default-74f4b96cdc-gnnfv" (UID: "9998d8a2-ea82-4b1f-aae9-554e79db6264") : configmap references non-existent config key: service-ca.crt Jul 02 19:08:29.443964 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:29.443835 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 19:08:29.443964 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:29.443855 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 19:08:29.443964 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:29.443881 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls podName:c8db16c6-0045-462c-a896-b66e91f4beb1 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:45.443868696 +0000 UTC m=+68.246684125 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-mlg8q" (UID: "c8db16c6-0045-462c-a896-b66e91f4beb1") : secret "cluster-monitoring-operator-tls" not found Jul 02 19:08:29.443964 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:29.443840 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 19:08:29.443964 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:29.443900 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f6886b98f-9sg6d: secret "image-registry-tls" not found Jul 02 19:08:29.443964 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:29.443906 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs podName:9998d8a2-ea82-4b1f-aae9-554e79db6264 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:45.443891716 +0000 UTC m=+68.246707157 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs") pod "router-default-74f4b96cdc-gnnfv" (UID: "9998d8a2-ea82-4b1f-aae9-554e79db6264") : secret "router-metrics-certs-default" not found Jul 02 19:08:29.443964 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:29.443921 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls podName:8b744daf-9c04-4bb1-bfe7-babae8745b4a nodeName:}" failed. No retries permitted until 2026-07-02 19:08:45.443914535 +0000 UTC m=+68.246729999 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls") pod "image-registry-7f6886b98f-9sg6d" (UID: "8b744daf-9c04-4bb1-bfe7-babae8745b4a") : secret "image-registry-tls" not found Jul 02 19:08:29.746429 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:29.746335 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:29.746569 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:29.746494 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:08:29.746569 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:08:29.746551 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls podName:b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:45.746536153 +0000 UTC m=+68.549351590 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls") pod "dns-default-24ghv" (UID: "b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71") : secret "dns-default-metrics-tls" not found Jul 02 19:08:30.132936 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:30.132890 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:30.416797 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:30.416710 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:30.515789 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:30.515752 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:30.617023 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:30.616993 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:30.756209 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:30.756125 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:30.933031 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:30.932992 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:31.136079 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:31.136053 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:32.302046 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:32.302012 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:32.512119 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:32.512086 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:32.700451 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:32.700414 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:32.922135 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:32.922101 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:33.101181 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:33.101141 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:33.811794 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:33.811763 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:34.095969 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:34.095879 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:34.300214 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:34.300166 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:34.498220 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:34.498147 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:34.697358 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:34.697325 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:34.797800 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:34.797725 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:34.957207 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:34.957176 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:36.096641 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:36.096602 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:36.097008 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:36.096656 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:36.097233 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:36.097214 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d12a1023-4e31-4eb4-9aab-0b737eb11f51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:36.098794 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:36.098778 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d12a1023-4e31-4eb4-9aab-0b737eb11f51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p47cx\" (UID: \"d12a1023-4e31-4eb4-9aab-0b737eb11f51\") " pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:36.210593 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:36.210561 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p47cx" Jul 02 19:08:36.334112 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:36.334084 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p47cx"] Jul 02 19:08:36.336792 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:08:36.336766 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd12a1023_4e31_4eb4_9aab_0b737eb11f51.slice/crio-e4d868ea0704e16976bc98aa36e6afe12ec1d285eba2a988eb9df79acfebba5b WatchSource:0}: Error finding container e4d868ea0704e16976bc98aa36e6afe12ec1d285eba2a988eb9df79acfebba5b: Status 404 returned error can't find the container with id e4d868ea0704e16976bc98aa36e6afe12ec1d285eba2a988eb9df79acfebba5b Jul 02 19:08:37.001633 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:37.001600 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p47cx" event={"ID":"d12a1023-4e31-4eb4-9aab-0b737eb11f51","Type":"ContainerStarted","Data":"e4d868ea0704e16976bc98aa36e6afe12ec1d285eba2a988eb9df79acfebba5b"} Jul 02 19:08:37.915639 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:37.915609 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7h54r" Jul 02 19:08:41.015511 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:41.015482 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p47cx" event={"ID":"d12a1023-4e31-4eb4-9aab-0b737eb11f51","Type":"ContainerStarted","Data":"ea9c684103201434545ddfa7f21cb36fce3cf34973d920be29032bd26c0eb70f"} Jul 02 19:08:41.015813 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:41.015534 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p47cx" event={"ID":"d12a1023-4e31-4eb4-9aab-0b737eb11f51","Type":"ContainerStarted","Data":"b3e2a31ec6469f479f3db85b10ac90939852ff6bb6934c29aa749166a8c62878"} Jul 02 19:08:42.457955 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:42.457914 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5562\" (UniqueName: \"kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562\") pod \"network-check-target-9qvrm\" (UID: \"f43fc6d3-00d7-4153-aea4-423b1d40e67b\") " pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:08:42.460867 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:42.460843 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 19:08:42.471380 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:42.471357 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 19:08:42.482256 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:42.482225 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5562\" (UniqueName: \"kubernetes.io/projected/f43fc6d3-00d7-4153-aea4-423b1d40e67b-kube-api-access-q5562\") pod \"network-check-target-9qvrm\" (UID: \"f43fc6d3-00d7-4153-aea4-423b1d40e67b\") " pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:08:42.558907 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:42.558864 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret\") pod \"global-pull-secret-syncer-p2dbb\" (UID: \"7bd39ca6-f010-4e22-b7b9-04bf2acece57\") " pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:08:42.559090 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:42.558922 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs\") pod \"network-metrics-daemon-rjlz4\" (UID: \"04c1f8e7-1552-41f6-b1af-d4ba971e3f76\") " pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:08:42.561488 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:42.561460 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 19:08:42.562053 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:42.562031 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 19:08:42.572516 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:42.572491 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bd39ca6-f010-4e22-b7b9-04bf2acece57-original-pull-secret\") pod \"global-pull-secret-syncer-p2dbb\" (UID: \"7bd39ca6-f010-4e22-b7b9-04bf2acece57\") " pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:08:42.572626 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:42.572523 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04c1f8e7-1552-41f6-b1af-d4ba971e3f76-metrics-certs\") pod \"network-metrics-daemon-rjlz4\" (UID: \"04c1f8e7-1552-41f6-b1af-d4ba971e3f76\") " pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:08:42.635981 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:42.635954 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5rf5h\"" Jul 02 19:08:42.642867 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:42.642838 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-r7xgd\"" Jul 02 19:08:42.644519 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:42.644501 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:08:42.645754 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:42.645736 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2dbb" Jul 02 19:08:42.651450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:42.650985 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjlz4" Jul 02 19:08:43.473747 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:43.473721 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9qvrm"] Jul 02 19:08:43.476620 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:08:43.476592 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf43fc6d3_00d7_4153_aea4_423b1d40e67b.slice/crio-4762142df1ea18bf500071b6e982cc8dfd91f1a9166d54d904b4c563645809b3 WatchSource:0}: Error finding container 4762142df1ea18bf500071b6e982cc8dfd91f1a9166d54d904b4c563645809b3: Status 404 returned error can't find the container with id 4762142df1ea18bf500071b6e982cc8dfd91f1a9166d54d904b4c563645809b3 Jul 02 19:08:43.693971 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:43.693617 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p2dbb"] Jul 02 19:08:43.696741 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:43.696718 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rjlz4"] Jul 02 19:08:43.696964 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:08:43.696945 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bd39ca6_f010_4e22_b7b9_04bf2acece57.slice/crio-c8570402821031c229044700aed8bbf7a7166cfa18c4eb5bcfce7de88b4a2e97 WatchSource:0}: Error finding container c8570402821031c229044700aed8bbf7a7166cfa18c4eb5bcfce7de88b4a2e97: Status 404 returned error can't find the container with id c8570402821031c229044700aed8bbf7a7166cfa18c4eb5bcfce7de88b4a2e97 Jul 02 19:08:43.699907 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:08:43.699880 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04c1f8e7_1552_41f6_b1af_d4ba971e3f76.slice/crio-2e9240cab97905f5392b010d8a372015faa981dc7afbef74d7af1c607ed97278 WatchSource:0}: Error finding container 2e9240cab97905f5392b010d8a372015faa981dc7afbef74d7af1c607ed97278: Status 404 returned error can't find the container with id 2e9240cab97905f5392b010d8a372015faa981dc7afbef74d7af1c607ed97278 Jul 02 19:08:44.028613 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:44.028532 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p47cx" event={"ID":"d12a1023-4e31-4eb4-9aab-0b737eb11f51","Type":"ContainerStarted","Data":"a5d4532023a47775b78dddc4bb735744dcf9a53eb1438cde9bcc0defb8918daa"} Jul 02 19:08:44.029867 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:44.029830 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p2dbb" event={"ID":"7bd39ca6-f010-4e22-b7b9-04bf2acece57","Type":"ContainerStarted","Data":"c8570402821031c229044700aed8bbf7a7166cfa18c4eb5bcfce7de88b4a2e97"} Jul 02 19:08:44.031072 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:44.031046 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rjlz4" event={"ID":"04c1f8e7-1552-41f6-b1af-d4ba971e3f76","Type":"ContainerStarted","Data":"2e9240cab97905f5392b010d8a372015faa981dc7afbef74d7af1c607ed97278"} Jul 02 19:08:44.032342 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:44.032315 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9qvrm" event={"ID":"f43fc6d3-00d7-4153-aea4-423b1d40e67b","Type":"ContainerStarted","Data":"4762142df1ea18bf500071b6e982cc8dfd91f1a9166d54d904b4c563645809b3"} Jul 02 19:08:44.047893 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:44.047840 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-p47cx" podStartSLOduration=17.116762674 podStartE2EDuration="24.047819983s" totalImagesPullingTime="6.931057309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:08:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:44.046230959 +0000 UTC m=+66.849046434" watchObservedRunningTime="2026-07-02 19:08:44.047819983 +0000 UTC m=+66.850635436" Jul 02 19:08:45.484074 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.483988 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:45.484074 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.484052 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:45.484584 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.484121 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:45.484584 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.484153 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mlg8q\" (UID: \"c8db16c6-0045-462c-a896-b66e91f4beb1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" Jul 02 19:08:45.485125 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.485086 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9998d8a2-ea82-4b1f-aae9-554e79db6264-service-ca-bundle\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:45.487278 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.487254 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls\") pod \"image-registry-7f6886b98f-9sg6d\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:45.488212 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.488188 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c8db16c6-0045-462c-a896-b66e91f4beb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mlg8q\" (UID: \"c8db16c6-0045-462c-a896-b66e91f4beb1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" Jul 02 19:08:45.488709 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.488662 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9998d8a2-ea82-4b1f-aae9-554e79db6264-metrics-certs\") pod \"router-default-74f4b96cdc-gnnfv\" (UID: \"9998d8a2-ea82-4b1f-aae9-554e79db6264\") " pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:45.675872 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.675837 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-fjjwr\"" Jul 02 19:08:45.683290 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.683264 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-d4gw7\"" Jul 02 19:08:45.684322 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.684300 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:45.687449 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.687429 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-l2lvj\"" Jul 02 19:08:45.691473 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.691452 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:45.695711 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.695551 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" Jul 02 19:08:45.786722 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.786189 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:45.789936 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.789875 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71-metrics-tls\") pod \"dns-default-24ghv\" (UID: \"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71\") " pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:45.857893 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.857863 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74f4b96cdc-gnnfv"] Jul 02 19:08:45.860931 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:08:45.860897 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9998d8a2_ea82_4b1f_aae9_554e79db6264.slice/crio-a7ad22c0e375e9e8f7ea397187dd1e8a971473fa78bf122887d003ac86a87bf2 WatchSource:0}: Error finding container a7ad22c0e375e9e8f7ea397187dd1e8a971473fa78bf122887d003ac86a87bf2: Status 404 returned error can't find the container with id a7ad22c0e375e9e8f7ea397187dd1e8a971473fa78bf122887d003ac86a87bf2 Jul 02 19:08:45.962372 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.962348 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-45sjn\"" Jul 02 19:08:45.971228 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:45.971201 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:46.041967 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:46.041877 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rjlz4" event={"ID":"04c1f8e7-1552-41f6-b1af-d4ba971e3f76","Type":"ContainerStarted","Data":"deb10edf776a1906630eb6f49003174adea87c31452055ca1d0cc57f0c9c6cb2"} Jul 02 19:08:46.041967 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:46.041943 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rjlz4" event={"ID":"04c1f8e7-1552-41f6-b1af-d4ba971e3f76","Type":"ContainerStarted","Data":"a20be7ad0dd4e9d08b9f1baf5a582e1b5f7375a7bf89f8f608bbc5cdc3af8237"} Jul 02 19:08:46.043880 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:46.043845 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" event={"ID":"9998d8a2-ea82-4b1f-aae9-554e79db6264","Type":"ContainerStarted","Data":"3ba4084801a7e22f46d54ed53e73d5b6ce5b04aa2a273b80a42551d0fbb92307"} Jul 02 19:08:46.043985 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:46.043892 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" event={"ID":"9998d8a2-ea82-4b1f-aae9-554e79db6264","Type":"ContainerStarted","Data":"a7ad22c0e375e9e8f7ea397187dd1e8a971473fa78bf122887d003ac86a87bf2"} Jul 02 19:08:46.078708 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:46.078652 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" podStartSLOduration=38.078630026 podStartE2EDuration="38.078630026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:46.077391081 +0000 UTC m=+68.880206533" watchObservedRunningTime="2026-07-02 19:08:46.078630026 +0000 UTC m=+68.881445476" Jul 02 19:08:46.079315 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:46.079276 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rjlz4" podStartSLOduration=66.717671993 podStartE2EDuration="1m8.079262593s" totalImagesPullingTime="1.3615906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:46.059332646 +0000 UTC m=+68.862148098" watchObservedRunningTime="2026-07-02 19:08:46.079262593 +0000 UTC m=+68.882078043" Jul 02 19:08:46.091216 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:46.091140 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q"] Jul 02 19:08:46.097032 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:46.097006 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f6886b98f-9sg6d"] Jul 02 19:08:46.685472 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:46.685443 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:46.688634 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:46.688606 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:47.048527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:47.048389 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:47.049627 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:47.049606 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-74f4b96cdc-gnnfv" Jul 02 19:08:47.109818 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:08:47.109786 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8db16c6_0045_462c_a896_b66e91f4beb1.slice/crio-3bbf309f5a87db5f46749697da35d7dd76e34908f87ceda397e3ed813255a169 WatchSource:0}: Error finding container 3bbf309f5a87db5f46749697da35d7dd76e34908f87ceda397e3ed813255a169: Status 404 returned error can't find the container with id 3bbf309f5a87db5f46749697da35d7dd76e34908f87ceda397e3ed813255a169 Jul 02 19:08:47.110336 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:08:47.110313 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b744daf_9c04_4bb1_bfe7_babae8745b4a.slice/crio-d19b75203faa09a7c7411ffd3f9e64d13ba4bdb7335b9c4b8d8fea51092909c8 WatchSource:0}: Error finding container d19b75203faa09a7c7411ffd3f9e64d13ba4bdb7335b9c4b8d8fea51092909c8: Status 404 returned error can't find the container with id d19b75203faa09a7c7411ffd3f9e64d13ba4bdb7335b9c4b8d8fea51092909c8 Jul 02 19:08:47.239360 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:47.239338 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-24ghv"] Jul 02 19:08:47.242273 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:08:47.242252 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb31b1e5b_c80b_47ca_b7f2_c9ad31a4ad71.slice/crio-1f495ed1db756c8c35e849ba0750629630baa0ffda64f28838c49d60cb5ca353 WatchSource:0}: Error finding container 1f495ed1db756c8c35e849ba0750629630baa0ffda64f28838c49d60cb5ca353: Status 404 returned error can't find the container with id 1f495ed1db756c8c35e849ba0750629630baa0ffda64f28838c49d60cb5ca353 Jul 02 19:08:48.013630 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.013600 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h"] Jul 02 19:08:48.050843 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.050812 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h"] Jul 02 19:08:48.051038 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.050972 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.054277 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.054253 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 02 19:08:48.055214 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.055063 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 02 19:08:48.055214 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.055079 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 02 19:08:48.055214 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.055100 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 02 19:08:48.055456 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.055428 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" event={"ID":"8b744daf-9c04-4bb1-bfe7-babae8745b4a","Type":"ContainerStarted","Data":"36a41902cc09bac7a681a39d6bfe94bea0af277ac89ea24b63a1d37fc4df89ed"} Jul 02 19:08:48.055526 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.055480 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" event={"ID":"8b744daf-9c04-4bb1-bfe7-babae8745b4a","Type":"ContainerStarted","Data":"d19b75203faa09a7c7411ffd3f9e64d13ba4bdb7335b9c4b8d8fea51092909c8"} Jul 02 19:08:48.055584 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.055567 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:08:48.056283 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.056259 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 02 19:08:48.057691 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.056909 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 02 19:08:48.057691 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.057260 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 02 19:08:48.063322 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.063288 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-24ghv" event={"ID":"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71","Type":"ContainerStarted","Data":"1f495ed1db756c8c35e849ba0750629630baa0ffda64f28838c49d60cb5ca353"} Jul 02 19:08:48.064687 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.064656 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" event={"ID":"c8db16c6-0045-462c-a896-b66e91f4beb1","Type":"ContainerStarted","Data":"3bbf309f5a87db5f46749697da35d7dd76e34908f87ceda397e3ed813255a169"} Jul 02 19:08:48.113583 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.113553 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-fng8v"] Jul 02 19:08:48.132296 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.132269 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-z9kqm"] Jul 02 19:08:48.132475 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.132410 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fng8v" Jul 02 19:08:48.143306 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.143248 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 02 19:08:48.144629 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.144584 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 02 19:08:48.145174 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.145153 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-l8l7c\"" Jul 02 19:08:48.146372 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.146322 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" podStartSLOduration=69.146307388 podStartE2EDuration="1m9.146307388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:07:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:48.14420103 +0000 UTC m=+70.947016502" watchObservedRunningTime="2026-07-02 19:08:48.146307388 +0000 UTC m=+70.949122839" Jul 02 19:08:48.156497 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.156474 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-fng8v"] Jul 02 19:08:48.156619 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.156602 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-z9kqm" Jul 02 19:08:48.168978 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.168937 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-rm6z4\"" Jul 02 19:08:48.170307 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.170289 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-z9kqm"] Jul 02 19:08:48.175497 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.175475 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 19:08:48.175610 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.175499 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 19:08:48.207733 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.207506 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9faa5a77-92a2-4793-a2a5-56abdd584e46-hub\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.207733 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.207717 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9faa5a77-92a2-4793-a2a5-56abdd584e46-ca\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.207916 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.207822 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9faa5a77-92a2-4793-a2a5-56abdd584e46-additional-service-ca\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.207916 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.207859 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d749313-98ec-4c39-9421-2f888d58af9c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fng8v\" (UID: \"9d749313-98ec-4c39-9421-2f888d58af9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fng8v" Jul 02 19:08:48.207916 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.207903 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vl9v\" (UniqueName: \"kubernetes.io/projected/9faa5a77-92a2-4793-a2a5-56abdd584e46-kube-api-access-2vl9v\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.208009 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.207952 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9d749313-98ec-4c39-9421-2f888d58af9c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-fng8v\" (UID: \"9d749313-98ec-4c39-9421-2f888d58af9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fng8v" Jul 02 19:08:48.208009 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.207989 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9faa5a77-92a2-4793-a2a5-56abdd584e46-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.208154 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.208127 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9faa5a77-92a2-4793-a2a5-56abdd584e46-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.309344 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.309315 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9faa5a77-92a2-4793-a2a5-56abdd584e46-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.309524 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.309355 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9faa5a77-92a2-4793-a2a5-56abdd584e46-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.309524 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.309424 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9faa5a77-92a2-4793-a2a5-56abdd584e46-hub\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.309524 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.309458 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9faa5a77-92a2-4793-a2a5-56abdd584e46-ca\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.309524 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.309506 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9faa5a77-92a2-4793-a2a5-56abdd584e46-additional-service-ca\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.309744 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.309537 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d749313-98ec-4c39-9421-2f888d58af9c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fng8v\" (UID: \"9d749313-98ec-4c39-9421-2f888d58af9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fng8v" Jul 02 19:08:48.309744 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.309572 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vl9v\" (UniqueName: \"kubernetes.io/projected/9faa5a77-92a2-4793-a2a5-56abdd584e46-kube-api-access-2vl9v\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.309744 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.309603 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9d749313-98ec-4c39-9421-2f888d58af9c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-fng8v\" (UID: \"9d749313-98ec-4c39-9421-2f888d58af9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fng8v" Jul 02 19:08:48.309744 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.309633 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2rvd\" (UniqueName: \"kubernetes.io/projected/65e26e27-be9e-4f51-afb7-08a3aed67d9b-kube-api-access-t2rvd\") pod \"downloads-685bfb5f64-z9kqm\" (UID: \"65e26e27-be9e-4f51-afb7-08a3aed67d9b\") " pod="openshift-console/downloads-685bfb5f64-z9kqm" Jul 02 19:08:48.310631 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.310573 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9faa5a77-92a2-4793-a2a5-56abdd584e46-additional-service-ca\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.310727 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.310710 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9d749313-98ec-4c39-9421-2f888d58af9c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-fng8v\" (UID: \"9d749313-98ec-4c39-9421-2f888d58af9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fng8v" Jul 02 19:08:48.312345 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.312318 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9faa5a77-92a2-4793-a2a5-56abdd584e46-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.312501 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.312466 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9faa5a77-92a2-4793-a2a5-56abdd584e46-ca\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.312632 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.312607 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d749313-98ec-4c39-9421-2f888d58af9c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fng8v\" (UID: \"9d749313-98ec-4c39-9421-2f888d58af9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fng8v" Jul 02 19:08:48.312709 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.312632 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9faa5a77-92a2-4793-a2a5-56abdd584e46-hub\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.312709 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.312674 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9faa5a77-92a2-4793-a2a5-56abdd584e46-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.320337 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.320290 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vl9v\" (UniqueName: \"kubernetes.io/projected/9faa5a77-92a2-4793-a2a5-56abdd584e46-kube-api-access-2vl9v\") pod \"cluster-proxy-proxy-agent-99644c47-crg6h\" (UID: \"9faa5a77-92a2-4793-a2a5-56abdd584e46\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.373305 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.373275 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" Jul 02 19:08:48.410433 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.410390 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2rvd\" (UniqueName: \"kubernetes.io/projected/65e26e27-be9e-4f51-afb7-08a3aed67d9b-kube-api-access-t2rvd\") pod \"downloads-685bfb5f64-z9kqm\" (UID: \"65e26e27-be9e-4f51-afb7-08a3aed67d9b\") " pod="openshift-console/downloads-685bfb5f64-z9kqm" Jul 02 19:08:48.419695 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.419671 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2rvd\" (UniqueName: \"kubernetes.io/projected/65e26e27-be9e-4f51-afb7-08a3aed67d9b-kube-api-access-t2rvd\") pod \"downloads-685bfb5f64-z9kqm\" (UID: \"65e26e27-be9e-4f51-afb7-08a3aed67d9b\") " pod="openshift-console/downloads-685bfb5f64-z9kqm" Jul 02 19:08:48.444435 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.444406 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fng8v" Jul 02 19:08:48.467263 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:48.467239 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-z9kqm" Jul 02 19:08:49.024154 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:49.024087 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h"] Jul 02 19:08:49.068839 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:49.068801 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9qvrm" event={"ID":"f43fc6d3-00d7-4153-aea4-423b1d40e67b","Type":"ContainerStarted","Data":"3ae259ab243f0c0e9c1364ee516401de7061d4a9a2832667b67d535344f9d622"} Jul 02 19:08:49.069159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:49.069126 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:08:49.085383 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:49.085236 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9qvrm" podStartSLOduration=65.695412549 podStartE2EDuration="1m11.085216484s" totalImagesPullingTime="5.389803935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:49.084613333 +0000 UTC m=+71.887428783" watchObservedRunningTime="2026-07-02 19:08:49.085216484 +0000 UTC m=+71.888031936" Jul 02 19:08:49.193254 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:08:49.193190 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9faa5a77_92a2_4793_a2a5_56abdd584e46.slice/crio-4254a19a26357c8ba08eae6fe34bc30f9ad088df47a17b94b48036a01c9a7122 WatchSource:0}: Error finding container 4254a19a26357c8ba08eae6fe34bc30f9ad088df47a17b94b48036a01c9a7122: Status 404 returned error can't find the container with id 4254a19a26357c8ba08eae6fe34bc30f9ad088df47a17b94b48036a01c9a7122 Jul 02 19:08:49.249977 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:49.249954 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-fng8v"] Jul 02 19:08:49.252834 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:08:49.252808 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d749313_98ec_4c39_9421_2f888d58af9c.slice/crio-526b00ded72b8e441edcbb2bfb0ba097c8f9eee6b0119287c6e7ec943283aba5 WatchSource:0}: Error finding container 526b00ded72b8e441edcbb2bfb0ba097c8f9eee6b0119287c6e7ec943283aba5: Status 404 returned error can't find the container with id 526b00ded72b8e441edcbb2bfb0ba097c8f9eee6b0119287c6e7ec943283aba5 Jul 02 19:08:49.253464 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:49.253445 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-z9kqm"] Jul 02 19:08:49.255778 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:08:49.255761 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e26e27_be9e_4f51_afb7_08a3aed67d9b.slice/crio-b87fb7f7d87a726bff04679593f2cb99744ef228a76b32e502785c4cf122d196 WatchSource:0}: Error finding container b87fb7f7d87a726bff04679593f2cb99744ef228a76b32e502785c4cf122d196: Status 404 returned error can't find the container with id b87fb7f7d87a726bff04679593f2cb99744ef228a76b32e502785c4cf122d196 Jul 02 19:08:50.080123 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:50.080080 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p2dbb" event={"ID":"7bd39ca6-f010-4e22-b7b9-04bf2acece57","Type":"ContainerStarted","Data":"31c19ef19aa2b6289e8861e88f4870ab460ea8222c7eaf76d6435a1ae4999ce4"} Jul 02 19:08:50.082744 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:50.082710 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" event={"ID":"9faa5a77-92a2-4793-a2a5-56abdd584e46","Type":"ContainerStarted","Data":"4254a19a26357c8ba08eae6fe34bc30f9ad088df47a17b94b48036a01c9a7122"} Jul 02 19:08:50.085040 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:50.084166 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fng8v" event={"ID":"9d749313-98ec-4c39-9421-2f888d58af9c","Type":"ContainerStarted","Data":"526b00ded72b8e441edcbb2bfb0ba097c8f9eee6b0119287c6e7ec943283aba5"} Jul 02 19:08:50.087545 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:50.087491 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-z9kqm" event={"ID":"65e26e27-be9e-4f51-afb7-08a3aed67d9b","Type":"ContainerStarted","Data":"b87fb7f7d87a726bff04679593f2cb99744ef228a76b32e502785c4cf122d196"} Jul 02 19:08:50.099991 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:50.099633 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-p2dbb" podStartSLOduration=66.564005591 podStartE2EDuration="1m12.099615976s" totalImagesPullingTime="5.535610385s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:50.09702153 +0000 UTC m=+72.899837005" watchObservedRunningTime="2026-07-02 19:08:50.099615976 +0000 UTC m=+72.902431425" Jul 02 19:08:55.111898 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:55.111862 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-24ghv" event={"ID":"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71","Type":"ContainerStarted","Data":"8114a6ee5735fdf160b4165d6ea321c8f4f509e8e913bf8081ef8fc8762342b2"} Jul 02 19:08:55.112368 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:55.111954 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-24ghv" event={"ID":"b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71","Type":"ContainerStarted","Data":"4c4a9a7097f4d162b40c1e2a21e72ba4e0ff2f782e4065fb5000e53614f4ca84"} Jul 02 19:08:55.112368 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:55.111983 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-24ghv" Jul 02 19:08:55.113734 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:55.113706 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" event={"ID":"c8db16c6-0045-462c-a896-b66e91f4beb1","Type":"ContainerStarted","Data":"3e403e8c57560fcc59aa6d8447d1e9f44e6c1f806ac5afe2c91e1d3eb3c10cc9"} Jul 02 19:08:55.116516 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:55.116492 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" event={"ID":"9faa5a77-92a2-4793-a2a5-56abdd584e46","Type":"ContainerStarted","Data":"63883d953ae73e2c391b1bd263f9aec076a8d7023f46ab3d0196d816035bacb3"} Jul 02 19:08:55.116637 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:55.116528 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" event={"ID":"9faa5a77-92a2-4793-a2a5-56abdd584e46","Type":"ContainerStarted","Data":"f7ddae93c3241d94714192f4e1efac6c57f83e1420ba680d1e49147012b409c7"} Jul 02 19:08:55.116637 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:55.116545 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" event={"ID":"9faa5a77-92a2-4793-a2a5-56abdd584e46","Type":"ContainerStarted","Data":"ea4997a7ef32a6f794011bf3e22bbc1be70e27ee6c2c51dcc1d45a8b7883b423"} Jul 02 19:08:55.118364 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:55.118304 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fng8v" event={"ID":"9d749313-98ec-4c39-9421-2f888d58af9c","Type":"ContainerStarted","Data":"d1ee07b2344a3c0c473c1008bce1900c15f167d7904a051ff7e7e8cf7a103884"} Jul 02 19:08:55.145382 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:55.145337 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-24ghv" podStartSLOduration=35.171387583 podStartE2EDuration="42.145323446s" totalImagesPullingTime="6.973935863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:55.143131444 +0000 UTC m=+77.945946895" watchObservedRunningTime="2026-07-02 19:08:55.145323446 +0000 UTC m=+77.948138896" Jul 02 19:08:55.160700 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:55.160638 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mlg8q" podStartSLOduration=40.055020885 podStartE2EDuration="47.160622848s" totalImagesPullingTime="7.105601963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:55.159501616 +0000 UTC m=+77.962317080" watchObservedRunningTime="2026-07-02 19:08:55.160622848 +0000 UTC m=+77.963438299" Jul 02 19:08:55.180644 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:55.180603 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-99644c47-crg6h" podStartSLOduration=3.104714369 podStartE2EDuration="8.180589757s" totalImagesPullingTime="5.075875388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:55.180294689 +0000 UTC m=+77.983110140" watchObservedRunningTime="2026-07-02 19:08:55.180589757 +0000 UTC m=+77.983405208" Jul 02 19:08:55.195696 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:08:55.195654 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fng8v" podStartSLOduration=2.229102801 podStartE2EDuration="7.195640531s" totalImagesPullingTime="4.96653773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:55.194483628 +0000 UTC m=+77.997299078" watchObservedRunningTime="2026-07-02 19:08:55.195640531 +0000 UTC m=+77.998455984" Jul 02 19:09:05.126685 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:05.126547 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-24ghv" Jul 02 19:09:05.696746 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:05.696709 2578 patch_prober.go:28] interesting pod/image-registry-7f6886b98f-9sg6d container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 02 19:09:05.696929 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:05.696771 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" podUID="8b744daf-9c04-4bb1-bfe7-babae8745b4a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 19:09:08.173063 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:08.172986 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-z9kqm" event={"ID":"65e26e27-be9e-4f51-afb7-08a3aed67d9b","Type":"ContainerStarted","Data":"16dbd37daeee071c09d3c7c76e44beac282a28682320964a26a4a3ade2d76472"} Jul 02 19:09:08.173522 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:08.173178 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-z9kqm" Jul 02 19:09:08.181505 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:08.181476 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-z9kqm" Jul 02 19:09:08.201002 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:08.200944 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-z9kqm" podStartSLOduration=1.6059571830000001 podStartE2EDuration="20.200926846s" totalImagesPullingTime="18.594969663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:09:08.193201551 +0000 UTC m=+90.996017004" watchObservedRunningTime="2026-07-02 19:09:08.200926846 +0000 UTC m=+91.003742302" Jul 02 19:09:09.073929 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:09.073894 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:09:09.921677 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:09.921644 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f6886b98f-9sg6d"] Jul 02 19:09:11.431092 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.431056 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fvcpp"] Jul 02 19:09:11.455812 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.455785 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.461223 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.461061 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 19:09:11.461594 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.461567 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 19:09:11.461750 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.461640 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jgkx2\"" Jul 02 19:09:11.462345 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.462327 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 19:09:11.462634 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.462605 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 19:09:11.593604 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.593569 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/93442352-1882-45bc-b2f9-e0ee72908d12-node-exporter-wtmp\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.593778 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.593612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/93442352-1882-45bc-b2f9-e0ee72908d12-root\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.593778 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.593643 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93442352-1882-45bc-b2f9-e0ee72908d12-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.593778 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.593668 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93442352-1882-45bc-b2f9-e0ee72908d12-sys\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.593778 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.593730 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/93442352-1882-45bc-b2f9-e0ee72908d12-node-exporter-textfile\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.593778 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.593748 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/93442352-1882-45bc-b2f9-e0ee72908d12-node-exporter-tls\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.593778 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.593772 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93442352-1882-45bc-b2f9-e0ee72908d12-metrics-client-ca\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.594093 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.593849 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bn8r\" (UniqueName: \"kubernetes.io/projected/93442352-1882-45bc-b2f9-e0ee72908d12-kube-api-access-9bn8r\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.594093 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.593912 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/93442352-1882-45bc-b2f9-e0ee72908d12-node-exporter-accelerators-collector-config\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.695098 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.695021 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/93442352-1882-45bc-b2f9-e0ee72908d12-node-exporter-accelerators-collector-config\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.695098 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.695063 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/93442352-1882-45bc-b2f9-e0ee72908d12-node-exporter-wtmp\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.695098 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.695090 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/93442352-1882-45bc-b2f9-e0ee72908d12-root\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.695375 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.695153 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/93442352-1882-45bc-b2f9-e0ee72908d12-root\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.695375 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.695188 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93442352-1882-45bc-b2f9-e0ee72908d12-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.695375 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.695228 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/93442352-1882-45bc-b2f9-e0ee72908d12-node-exporter-wtmp\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.695375 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.695232 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93442352-1882-45bc-b2f9-e0ee72908d12-sys\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.695375 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.695271 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93442352-1882-45bc-b2f9-e0ee72908d12-sys\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.695375 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.695332 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/93442352-1882-45bc-b2f9-e0ee72908d12-node-exporter-textfile\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.695375 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.695361 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/93442352-1882-45bc-b2f9-e0ee72908d12-node-exporter-tls\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.695739 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.695387 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93442352-1882-45bc-b2f9-e0ee72908d12-metrics-client-ca\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.695739 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.695450 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bn8r\" (UniqueName: \"kubernetes.io/projected/93442352-1882-45bc-b2f9-e0ee72908d12-kube-api-access-9bn8r\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.695739 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.695659 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/93442352-1882-45bc-b2f9-e0ee72908d12-node-exporter-textfile\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.695888 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.695745 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/93442352-1882-45bc-b2f9-e0ee72908d12-node-exporter-accelerators-collector-config\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.695944 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.695911 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93442352-1882-45bc-b2f9-e0ee72908d12-metrics-client-ca\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.697885 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.697862 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93442352-1882-45bc-b2f9-e0ee72908d12-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.698038 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.698014 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/93442352-1882-45bc-b2f9-e0ee72908d12-node-exporter-tls\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.704288 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.704266 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bn8r\" (UniqueName: \"kubernetes.io/projected/93442352-1882-45bc-b2f9-e0ee72908d12-kube-api-access-9bn8r\") pod \"node-exporter-fvcpp\" (UID: \"93442352-1882-45bc-b2f9-e0ee72908d12\") " pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.767328 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:11.767291 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fvcpp" Jul 02 19:09:11.806224 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:09:11.806184 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93442352_1882_45bc_b2f9_e0ee72908d12.slice/crio-8114d8aa0b91bbba6c25bca28faf8fe7030338824241394ad73e6453a9925cec WatchSource:0}: Error finding container 8114d8aa0b91bbba6c25bca28faf8fe7030338824241394ad73e6453a9925cec: Status 404 returned error can't find the container with id 8114d8aa0b91bbba6c25bca28faf8fe7030338824241394ad73e6453a9925cec Jul 02 19:09:12.190787 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:12.190741 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fvcpp" event={"ID":"93442352-1882-45bc-b2f9-e0ee72908d12","Type":"ContainerStarted","Data":"8114d8aa0b91bbba6c25bca28faf8fe7030338824241394ad73e6453a9925cec"} Jul 02 19:09:14.202295 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:14.202197 2578 generic.go:356] "Generic (PLEG): container finished" podID="93442352-1882-45bc-b2f9-e0ee72908d12" containerID="6a0ce22b40f5b57411b71f8ef93373d8530c21b9bd72c2cd071ea9da58f208fc" exitCode=0 Jul 02 19:09:14.202749 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:14.202287 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fvcpp" event={"ID":"93442352-1882-45bc-b2f9-e0ee72908d12","Type":"ContainerDied","Data":"6a0ce22b40f5b57411b71f8ef93373d8530c21b9bd72c2cd071ea9da58f208fc"} Jul 02 19:09:15.209604 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:15.209561 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fvcpp" event={"ID":"93442352-1882-45bc-b2f9-e0ee72908d12","Type":"ContainerStarted","Data":"4f22914a2c952477501ba7a755d7dcd54e99f25aed0018a941f8f5faed7a4c09"} Jul 02 19:09:15.210006 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:15.209633 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fvcpp" event={"ID":"93442352-1882-45bc-b2f9-e0ee72908d12","Type":"ContainerStarted","Data":"fcd62fed1035b292fd98da109ee2600a7520e7e4dd1ad4ec61c3e4c5413676d8"} Jul 02 19:09:21.095106 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:21.095077 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9qvrm" Jul 02 19:09:21.114805 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:21.114753 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fvcpp" podStartSLOduration=8.243634974 podStartE2EDuration="10.114738623s" totalImagesPullingTime="1.756499056s" totalInitContainerRuntime="114.604593ms" isStatefulPod=true podCreationTimestamp="2026-07-02 19:09:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:09:15.234098536 +0000 UTC m=+98.036913988" watchObservedRunningTime="2026-07-02 19:09:21.114738623 +0000 UTC m=+103.917554112" Jul 02 19:09:34.947282 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:34.947241 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" podUID="8b744daf-9c04-4bb1-bfe7-babae8745b4a" containerName="registry" containerID="cri-o://36a41902cc09bac7a681a39d6bfe94bea0af277ac89ea24b63a1d37fc4df89ed" gracePeriod=30 Jul 02 19:09:35.217018 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.216956 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:09:35.271861 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.271836 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b744daf-9c04-4bb1-bfe7-babae8745b4a-image-registry-private-configuration\") pod \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " Jul 02 19:09:35.271861 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.271865 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls\") pod \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " Jul 02 19:09:35.272061 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.271887 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-certificates\") pod \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " Jul 02 19:09:35.272061 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.271945 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvhgh\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-kube-api-access-vvhgh\") pod \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " Jul 02 19:09:35.272061 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.271970 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b744daf-9c04-4bb1-bfe7-babae8745b4a-trusted-ca\") pod \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " Jul 02 19:09:35.272061 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.272022 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b744daf-9c04-4bb1-bfe7-babae8745b4a-installation-pull-secrets\") pod \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " Jul 02 19:09:35.272061 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.272055 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b744daf-9c04-4bb1-bfe7-babae8745b4a-ca-trust-extracted\") pod \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " Jul 02 19:09:35.272298 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.272087 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-bound-sa-token\") pod \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\" (UID: \"8b744daf-9c04-4bb1-bfe7-babae8745b4a\") " Jul 02 19:09:35.272616 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.272536 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b744daf-9c04-4bb1-bfe7-babae8745b4a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8b744daf-9c04-4bb1-bfe7-babae8745b4a" (UID: "8b744daf-9c04-4bb1-bfe7-babae8745b4a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:09:35.272616 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.272434 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8b744daf-9c04-4bb1-bfe7-babae8745b4a" (UID: "8b744daf-9c04-4bb1-bfe7-babae8745b4a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:09:35.274734 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.274686 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8b744daf-9c04-4bb1-bfe7-babae8745b4a" (UID: "8b744daf-9c04-4bb1-bfe7-babae8745b4a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:09:35.274995 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.274968 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b744daf-9c04-4bb1-bfe7-babae8745b4a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8b744daf-9c04-4bb1-bfe7-babae8745b4a" (UID: "8b744daf-9c04-4bb1-bfe7-babae8745b4a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:09:35.275101 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.275018 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8b744daf-9c04-4bb1-bfe7-babae8745b4a" (UID: "8b744daf-9c04-4bb1-bfe7-babae8745b4a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:09:35.275157 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.275094 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b744daf-9c04-4bb1-bfe7-babae8745b4a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8b744daf-9c04-4bb1-bfe7-babae8745b4a" (UID: "8b744daf-9c04-4bb1-bfe7-babae8745b4a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:09:35.275250 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.275147 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-kube-api-access-vvhgh" (OuterVolumeSpecName: "kube-api-access-vvhgh") pod "8b744daf-9c04-4bb1-bfe7-babae8745b4a" (UID: "8b744daf-9c04-4bb1-bfe7-babae8745b4a"). InnerVolumeSpecName "kube-api-access-vvhgh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:09:35.278747 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.278722 2578 generic.go:356] "Generic (PLEG): container finished" podID="8b744daf-9c04-4bb1-bfe7-babae8745b4a" containerID="36a41902cc09bac7a681a39d6bfe94bea0af277ac89ea24b63a1d37fc4df89ed" exitCode=0 Jul 02 19:09:35.278836 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.278756 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" event={"ID":"8b744daf-9c04-4bb1-bfe7-babae8745b4a","Type":"ContainerDied","Data":"36a41902cc09bac7a681a39d6bfe94bea0af277ac89ea24b63a1d37fc4df89ed"} Jul 02 19:09:35.278836 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.278773 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" Jul 02 19:09:35.278836 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.278812 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f6886b98f-9sg6d" event={"ID":"8b744daf-9c04-4bb1-bfe7-babae8745b4a","Type":"ContainerDied","Data":"d19b75203faa09a7c7411ffd3f9e64d13ba4bdb7335b9c4b8d8fea51092909c8"} Jul 02 19:09:35.278952 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.278837 2578 scope.go:117] "RemoveContainer" containerID="36a41902cc09bac7a681a39d6bfe94bea0af277ac89ea24b63a1d37fc4df89ed" Jul 02 19:09:35.281523 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.281497 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b744daf-9c04-4bb1-bfe7-babae8745b4a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8b744daf-9c04-4bb1-bfe7-babae8745b4a" (UID: "8b744daf-9c04-4bb1-bfe7-babae8745b4a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:09:35.290009 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.289990 2578 scope.go:117] "RemoveContainer" containerID="36a41902cc09bac7a681a39d6bfe94bea0af277ac89ea24b63a1d37fc4df89ed" Jul 02 19:09:35.290267 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:09:35.290243 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36a41902cc09bac7a681a39d6bfe94bea0af277ac89ea24b63a1d37fc4df89ed\": container with ID starting with 36a41902cc09bac7a681a39d6bfe94bea0af277ac89ea24b63a1d37fc4df89ed not found: ID does not exist" containerID="36a41902cc09bac7a681a39d6bfe94bea0af277ac89ea24b63a1d37fc4df89ed" Jul 02 19:09:35.290312 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.290280 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36a41902cc09bac7a681a39d6bfe94bea0af277ac89ea24b63a1d37fc4df89ed"} err="failed to get container status \"36a41902cc09bac7a681a39d6bfe94bea0af277ac89ea24b63a1d37fc4df89ed\": rpc error: code = NotFound desc = could not find container \"36a41902cc09bac7a681a39d6bfe94bea0af277ac89ea24b63a1d37fc4df89ed\": container with ID starting with 36a41902cc09bac7a681a39d6bfe94bea0af277ac89ea24b63a1d37fc4df89ed not found: ID does not exist" Jul 02 19:09:35.372848 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.372816 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b744daf-9c04-4bb1-bfe7-babae8745b4a-installation-pull-secrets\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:09:35.372848 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.372844 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b744daf-9c04-4bb1-bfe7-babae8745b4a-ca-trust-extracted\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:09:35.372848 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.372854 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-bound-sa-token\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:09:35.373059 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.372864 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b744daf-9c04-4bb1-bfe7-babae8745b4a-image-registry-private-configuration\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:09:35.373059 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.372873 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-tls\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:09:35.373059 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.372881 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b744daf-9c04-4bb1-bfe7-babae8745b4a-registry-certificates\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:09:35.373059 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.372889 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvhgh\" (UniqueName: \"kubernetes.io/projected/8b744daf-9c04-4bb1-bfe7-babae8745b4a-kube-api-access-vvhgh\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:09:35.373059 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.372897 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b744daf-9c04-4bb1-bfe7-babae8745b4a-trusted-ca\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:09:35.599435 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.599410 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f6886b98f-9sg6d"] Jul 02 19:09:35.605202 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.605179 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7f6886b98f-9sg6d"] Jul 02 19:09:35.725861 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:35.725832 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b744daf-9c04-4bb1-bfe7-babae8745b4a" path="/var/lib/kubelet/pods/8b744daf-9c04-4bb1-bfe7-babae8745b4a/volumes" Jul 02 19:09:50.330272 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:50.330234 2578 generic.go:356] "Generic (PLEG): container finished" podID="2bb75416-de32-44da-b7ae-29678f1cc099" containerID="3a10270d089b3391cbb6f443aaa6f96c5789e93068b5650d1a5b012453f50f7a" exitCode=0 Jul 02 19:09:50.330695 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:50.330293 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" event={"ID":"2bb75416-de32-44da-b7ae-29678f1cc099","Type":"ContainerDied","Data":"3a10270d089b3391cbb6f443aaa6f96c5789e93068b5650d1a5b012453f50f7a"} Jul 02 19:09:50.330695 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:50.330612 2578 scope.go:117] "RemoveContainer" containerID="3a10270d089b3391cbb6f443aaa6f96c5789e93068b5650d1a5b012453f50f7a" Jul 02 19:09:51.335662 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:09:51.335626 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-xrtv2" event={"ID":"2bb75416-de32-44da-b7ae-29678f1cc099","Type":"ContainerStarted","Data":"bb2bc58db69d59bc8804d9c48e4f930ee70249cb6122c0627c80ab65f7dfe7bb"} Jul 02 19:12:10.772296 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:10.772205 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gvspk"] Jul 02 19:12:10.772908 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:10.772577 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b744daf-9c04-4bb1-bfe7-babae8745b4a" containerName="registry" Jul 02 19:12:10.772908 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:10.772598 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b744daf-9c04-4bb1-bfe7-babae8745b4a" containerName="registry" Jul 02 19:12:10.772908 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:10.772681 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b744daf-9c04-4bb1-bfe7-babae8745b4a" containerName="registry" Jul 02 19:12:10.775610 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:10.775590 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gvspk" Jul 02 19:12:10.778321 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:10.778303 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-rc9q7\"" Jul 02 19:12:10.778563 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:10.778548 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 02 19:12:10.778613 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:10.778555 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 02 19:12:10.807521 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:10.807500 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gvspk"] Jul 02 19:12:10.861094 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:10.861073 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/657b2985-65ed-4061-93af-9062cbee8bca-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-gvspk\" (UID: \"657b2985-65ed-4061-93af-9062cbee8bca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gvspk" Jul 02 19:12:10.861197 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:10.861122 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7j6g\" (UniqueName: \"kubernetes.io/projected/657b2985-65ed-4061-93af-9062cbee8bca-kube-api-access-x7j6g\") pod \"cert-manager-operator-controller-manager-55c444bf86-gvspk\" (UID: \"657b2985-65ed-4061-93af-9062cbee8bca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gvspk" Jul 02 19:12:10.962176 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:10.962137 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7j6g\" (UniqueName: \"kubernetes.io/projected/657b2985-65ed-4061-93af-9062cbee8bca-kube-api-access-x7j6g\") pod \"cert-manager-operator-controller-manager-55c444bf86-gvspk\" (UID: \"657b2985-65ed-4061-93af-9062cbee8bca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gvspk" Jul 02 19:12:10.962266 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:10.962192 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/657b2985-65ed-4061-93af-9062cbee8bca-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-gvspk\" (UID: \"657b2985-65ed-4061-93af-9062cbee8bca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gvspk" Jul 02 19:12:10.962538 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:10.962524 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/657b2985-65ed-4061-93af-9062cbee8bca-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-gvspk\" (UID: \"657b2985-65ed-4061-93af-9062cbee8bca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gvspk" Jul 02 19:12:10.973346 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:10.973315 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7j6g\" (UniqueName: \"kubernetes.io/projected/657b2985-65ed-4061-93af-9062cbee8bca-kube-api-access-x7j6g\") pod \"cert-manager-operator-controller-manager-55c444bf86-gvspk\" (UID: \"657b2985-65ed-4061-93af-9062cbee8bca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gvspk" Jul 02 19:12:11.083695 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:11.083666 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gvspk" Jul 02 19:12:11.213384 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:11.213359 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gvspk"] Jul 02 19:12:11.216373 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:12:11.216348 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod657b2985_65ed_4061_93af_9062cbee8bca.slice/crio-e54a2c73e900f3100863be1a7318fd85186b206c240fbea8790318fe4909b574 WatchSource:0}: Error finding container e54a2c73e900f3100863be1a7318fd85186b206c240fbea8790318fe4909b574: Status 404 returned error can't find the container with id e54a2c73e900f3100863be1a7318fd85186b206c240fbea8790318fe4909b574 Jul 02 19:12:11.796944 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:11.796907 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gvspk" event={"ID":"657b2985-65ed-4061-93af-9062cbee8bca","Type":"ContainerStarted","Data":"e54a2c73e900f3100863be1a7318fd85186b206c240fbea8790318fe4909b574"} Jul 02 19:12:14.811092 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:14.811058 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gvspk" event={"ID":"657b2985-65ed-4061-93af-9062cbee8bca","Type":"ContainerStarted","Data":"ecea230edcab8d65cc3d5147b5a6bf42f8b3f1839bfabb9e18dceb5129059c20"} Jul 02 19:12:14.835293 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:14.835246 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gvspk" podStartSLOduration=1.835235637 podStartE2EDuration="4.835227583s" totalImagesPullingTime="2.999991946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:14.835212956 +0000 UTC m=+277.638028431" watchObservedRunningTime="2026-07-02 19:12:14.835227583 +0000 UTC m=+277.638043034" Jul 02 19:12:15.369070 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.369035 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g"] Jul 02 19:12:15.372428 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.372391 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" Jul 02 19:12:15.375825 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.375804 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2sbhq\"" Jul 02 19:12:15.375928 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.375908 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 19:12:15.376584 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.376567 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 19:12:15.382351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.382328 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g"] Jul 02 19:12:15.496631 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.496600 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tthtx\" (UniqueName: \"kubernetes.io/projected/e68be034-0f4b-4839-be5a-12e8bf10eb9e-kube-api-access-tthtx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g\" (UID: \"e68be034-0f4b-4839-be5a-12e8bf10eb9e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" Jul 02 19:12:15.496793 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.496649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e68be034-0f4b-4839-be5a-12e8bf10eb9e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g\" (UID: \"e68be034-0f4b-4839-be5a-12e8bf10eb9e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" Jul 02 19:12:15.496793 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.496730 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e68be034-0f4b-4839-be5a-12e8bf10eb9e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g\" (UID: \"e68be034-0f4b-4839-be5a-12e8bf10eb9e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" Jul 02 19:12:15.597100 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.597067 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tthtx\" (UniqueName: \"kubernetes.io/projected/e68be034-0f4b-4839-be5a-12e8bf10eb9e-kube-api-access-tthtx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g\" (UID: \"e68be034-0f4b-4839-be5a-12e8bf10eb9e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" Jul 02 19:12:15.597100 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.597106 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e68be034-0f4b-4839-be5a-12e8bf10eb9e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g\" (UID: \"e68be034-0f4b-4839-be5a-12e8bf10eb9e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" Jul 02 19:12:15.597310 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.597135 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e68be034-0f4b-4839-be5a-12e8bf10eb9e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g\" (UID: \"e68be034-0f4b-4839-be5a-12e8bf10eb9e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" Jul 02 19:12:15.597504 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.597487 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e68be034-0f4b-4839-be5a-12e8bf10eb9e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g\" (UID: \"e68be034-0f4b-4839-be5a-12e8bf10eb9e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" Jul 02 19:12:15.597569 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.597506 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e68be034-0f4b-4839-be5a-12e8bf10eb9e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g\" (UID: \"e68be034-0f4b-4839-be5a-12e8bf10eb9e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" Jul 02 19:12:15.605813 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.605784 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tthtx\" (UniqueName: \"kubernetes.io/projected/e68be034-0f4b-4839-be5a-12e8bf10eb9e-kube-api-access-tthtx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g\" (UID: \"e68be034-0f4b-4839-be5a-12e8bf10eb9e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" Jul 02 19:12:15.681254 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.681188 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" Jul 02 19:12:15.805375 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.805352 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g"] Jul 02 19:12:15.807598 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:12:15.807569 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode68be034_0f4b_4839_be5a_12e8bf10eb9e.slice/crio-36c20f9658e29489de17d5a6674fbcd5e072018a783e633c99999ca05194f657 WatchSource:0}: Error finding container 36c20f9658e29489de17d5a6674fbcd5e072018a783e633c99999ca05194f657: Status 404 returned error can't find the container with id 36c20f9658e29489de17d5a6674fbcd5e072018a783e633c99999ca05194f657 Jul 02 19:12:15.816439 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:15.816411 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" event={"ID":"e68be034-0f4b-4839-be5a-12e8bf10eb9e","Type":"ContainerStarted","Data":"36c20f9658e29489de17d5a6674fbcd5e072018a783e633c99999ca05194f657"} Jul 02 19:12:23.858685 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:23.858650 2578 generic.go:356] "Generic (PLEG): container finished" podID="e68be034-0f4b-4839-be5a-12e8bf10eb9e" containerID="fc94646b9e25f8db30f540744dc45e6cbd91d61cd8a8c3e621c1366631e29d06" exitCode=0 Jul 02 19:12:23.859130 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:23.858699 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" event={"ID":"e68be034-0f4b-4839-be5a-12e8bf10eb9e","Type":"ContainerDied","Data":"fc94646b9e25f8db30f540744dc45e6cbd91d61cd8a8c3e621c1366631e29d06"} Jul 02 19:12:25.868962 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:25.868926 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" event={"ID":"e68be034-0f4b-4839-be5a-12e8bf10eb9e","Type":"ContainerStarted","Data":"94d0aa0a0c46210c0228c072a4184f1d90d529bf57f7e64d6604384598e9cd94"} Jul 02 19:12:26.874180 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:26.874146 2578 generic.go:356] "Generic (PLEG): container finished" podID="e68be034-0f4b-4839-be5a-12e8bf10eb9e" containerID="94d0aa0a0c46210c0228c072a4184f1d90d529bf57f7e64d6604384598e9cd94" exitCode=0 Jul 02 19:12:26.874668 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:26.874233 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" event={"ID":"e68be034-0f4b-4839-be5a-12e8bf10eb9e","Type":"ContainerDied","Data":"94d0aa0a0c46210c0228c072a4184f1d90d529bf57f7e64d6604384598e9cd94"} Jul 02 19:12:36.917449 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:36.917417 2578 generic.go:356] "Generic (PLEG): container finished" podID="e68be034-0f4b-4839-be5a-12e8bf10eb9e" containerID="7cd1ee76cf3ff5658096ca8210245e20f48ed28370901dadb36778498adccc4e" exitCode=0 Jul 02 19:12:36.917830 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:36.917488 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" event={"ID":"e68be034-0f4b-4839-be5a-12e8bf10eb9e","Type":"ContainerDied","Data":"7cd1ee76cf3ff5658096ca8210245e20f48ed28370901dadb36778498adccc4e"} Jul 02 19:12:37.632676 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:37.632639 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 19:12:38.048732 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:38.048708 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" Jul 02 19:12:38.072468 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:38.072441 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e68be034-0f4b-4839-be5a-12e8bf10eb9e-util\") pod \"e68be034-0f4b-4839-be5a-12e8bf10eb9e\" (UID: \"e68be034-0f4b-4839-be5a-12e8bf10eb9e\") " Jul 02 19:12:38.072625 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:38.072525 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tthtx\" (UniqueName: \"kubernetes.io/projected/e68be034-0f4b-4839-be5a-12e8bf10eb9e-kube-api-access-tthtx\") pod \"e68be034-0f4b-4839-be5a-12e8bf10eb9e\" (UID: \"e68be034-0f4b-4839-be5a-12e8bf10eb9e\") " Jul 02 19:12:38.072625 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:38.072592 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e68be034-0f4b-4839-be5a-12e8bf10eb9e-bundle\") pod \"e68be034-0f4b-4839-be5a-12e8bf10eb9e\" (UID: \"e68be034-0f4b-4839-be5a-12e8bf10eb9e\") " Jul 02 19:12:38.073104 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:38.073079 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e68be034-0f4b-4839-be5a-12e8bf10eb9e-bundle" (OuterVolumeSpecName: "bundle") pod "e68be034-0f4b-4839-be5a-12e8bf10eb9e" (UID: "e68be034-0f4b-4839-be5a-12e8bf10eb9e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:12:38.075412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:38.075364 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e68be034-0f4b-4839-be5a-12e8bf10eb9e-kube-api-access-tthtx" (OuterVolumeSpecName: "kube-api-access-tthtx") pod "e68be034-0f4b-4839-be5a-12e8bf10eb9e" (UID: "e68be034-0f4b-4839-be5a-12e8bf10eb9e"). InnerVolumeSpecName "kube-api-access-tthtx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:12:38.079498 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:38.079475 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e68be034-0f4b-4839-be5a-12e8bf10eb9e-util" (OuterVolumeSpecName: "util") pod "e68be034-0f4b-4839-be5a-12e8bf10eb9e" (UID: "e68be034-0f4b-4839-be5a-12e8bf10eb9e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:12:38.173688 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:38.173603 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tthtx\" (UniqueName: \"kubernetes.io/projected/e68be034-0f4b-4839-be5a-12e8bf10eb9e-kube-api-access-tthtx\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:12:38.173688 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:38.173636 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e68be034-0f4b-4839-be5a-12e8bf10eb9e-bundle\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:12:38.173688 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:38.173650 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e68be034-0f4b-4839-be5a-12e8bf10eb9e-util\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:12:38.926293 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:38.926262 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" event={"ID":"e68be034-0f4b-4839-be5a-12e8bf10eb9e","Type":"ContainerDied","Data":"36c20f9658e29489de17d5a6674fbcd5e072018a783e633c99999ca05194f657"} Jul 02 19:12:38.926293 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:38.926289 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzsm6g" Jul 02 19:12:38.926565 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:38.926309 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36c20f9658e29489de17d5a6674fbcd5e072018a783e633c99999ca05194f657" Jul 02 19:12:50.134632 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.134598 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8"] Jul 02 19:12:50.135098 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.134900 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e68be034-0f4b-4839-be5a-12e8bf10eb9e" containerName="util" Jul 02 19:12:50.135098 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.134913 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e68be034-0f4b-4839-be5a-12e8bf10eb9e" containerName="util" Jul 02 19:12:50.135098 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.134922 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e68be034-0f4b-4839-be5a-12e8bf10eb9e" containerName="pull" Jul 02 19:12:50.135098 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.134927 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e68be034-0f4b-4839-be5a-12e8bf10eb9e" containerName="pull" Jul 02 19:12:50.135098 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.134935 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e68be034-0f4b-4839-be5a-12e8bf10eb9e" containerName="extract" Jul 02 19:12:50.135098 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.134941 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e68be034-0f4b-4839-be5a-12e8bf10eb9e" containerName="extract" Jul 02 19:12:50.135098 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.134988 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e68be034-0f4b-4839-be5a-12e8bf10eb9e" containerName="extract" Jul 02 19:12:50.138685 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.138666 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" Jul 02 19:12:50.140978 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.140953 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 19:12:50.141093 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.140999 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2sbhq\"" Jul 02 19:12:50.141093 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.140953 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 19:12:50.146198 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.146175 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8"] Jul 02 19:12:50.157530 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.157503 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqp7r\" (UniqueName: \"kubernetes.io/projected/c4b31346-488b-48db-bbb5-f00522b53261-kube-api-access-wqp7r\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8\" (UID: \"c4b31346-488b-48db-bbb5-f00522b53261\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" Jul 02 19:12:50.157644 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.157550 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4b31346-488b-48db-bbb5-f00522b53261-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8\" (UID: \"c4b31346-488b-48db-bbb5-f00522b53261\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" Jul 02 19:12:50.157644 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.157577 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4b31346-488b-48db-bbb5-f00522b53261-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8\" (UID: \"c4b31346-488b-48db-bbb5-f00522b53261\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" Jul 02 19:12:50.258838 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.258805 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqp7r\" (UniqueName: \"kubernetes.io/projected/c4b31346-488b-48db-bbb5-f00522b53261-kube-api-access-wqp7r\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8\" (UID: \"c4b31346-488b-48db-bbb5-f00522b53261\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" Jul 02 19:12:50.258838 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.258844 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4b31346-488b-48db-bbb5-f00522b53261-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8\" (UID: \"c4b31346-488b-48db-bbb5-f00522b53261\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" Jul 02 19:12:50.259033 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.258861 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4b31346-488b-48db-bbb5-f00522b53261-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8\" (UID: \"c4b31346-488b-48db-bbb5-f00522b53261\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" Jul 02 19:12:50.259244 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.259219 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4b31346-488b-48db-bbb5-f00522b53261-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8\" (UID: \"c4b31346-488b-48db-bbb5-f00522b53261\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" Jul 02 19:12:50.259283 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.259247 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4b31346-488b-48db-bbb5-f00522b53261-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8\" (UID: \"c4b31346-488b-48db-bbb5-f00522b53261\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" Jul 02 19:12:50.266476 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.266454 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqp7r\" (UniqueName: \"kubernetes.io/projected/c4b31346-488b-48db-bbb5-f00522b53261-kube-api-access-wqp7r\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8\" (UID: \"c4b31346-488b-48db-bbb5-f00522b53261\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" Jul 02 19:12:50.448446 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.448351 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" Jul 02 19:12:50.574139 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.574110 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8"] Jul 02 19:12:50.576890 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:12:50.576862 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b31346_488b_48db_bbb5_f00522b53261.slice/crio-254e7bb67ace49e39a02753ba845ea3924a2e5f21a463e9e59441591ff26dc09 WatchSource:0}: Error finding container 254e7bb67ace49e39a02753ba845ea3924a2e5f21a463e9e59441591ff26dc09: Status 404 returned error can't find the container with id 254e7bb67ace49e39a02753ba845ea3924a2e5f21a463e9e59441591ff26dc09 Jul 02 19:12:50.975015 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.974978 2578 generic.go:356] "Generic (PLEG): container finished" podID="c4b31346-488b-48db-bbb5-f00522b53261" containerID="0ac31280fb587f6f4dfb4a2ef4a06a98658b177cbf5fd3603262b9d310926261" exitCode=0 Jul 02 19:12:50.975186 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.975044 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" event={"ID":"c4b31346-488b-48db-bbb5-f00522b53261","Type":"ContainerDied","Data":"0ac31280fb587f6f4dfb4a2ef4a06a98658b177cbf5fd3603262b9d310926261"} Jul 02 19:12:50.975186 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.975078 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" event={"ID":"c4b31346-488b-48db-bbb5-f00522b53261","Type":"ContainerStarted","Data":"254e7bb67ace49e39a02753ba845ea3924a2e5f21a463e9e59441591ff26dc09"} Jul 02 19:12:50.975520 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:50.975504 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 19:12:51.981356 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:51.981319 2578 generic.go:356] "Generic (PLEG): container finished" podID="c4b31346-488b-48db-bbb5-f00522b53261" containerID="d71400867dbb7be67b3c88e93568b670bbe5e2030493bce42dea5efece80f772" exitCode=0 Jul 02 19:12:51.981758 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:51.981366 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" event={"ID":"c4b31346-488b-48db-bbb5-f00522b53261","Type":"ContainerDied","Data":"d71400867dbb7be67b3c88e93568b670bbe5e2030493bce42dea5efece80f772"} Jul 02 19:12:52.987526 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:52.987490 2578 generic.go:356] "Generic (PLEG): container finished" podID="c4b31346-488b-48db-bbb5-f00522b53261" containerID="7646f15472e2e865da814590f6313a1139f4a72847f4caafa1e9350917c0910c" exitCode=0 Jul 02 19:12:52.987914 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:52.987531 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" event={"ID":"c4b31346-488b-48db-bbb5-f00522b53261","Type":"ContainerDied","Data":"7646f15472e2e865da814590f6313a1139f4a72847f4caafa1e9350917c0910c"} Jul 02 19:12:54.112260 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:54.112236 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" Jul 02 19:12:54.184675 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:54.184641 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4b31346-488b-48db-bbb5-f00522b53261-util\") pod \"c4b31346-488b-48db-bbb5-f00522b53261\" (UID: \"c4b31346-488b-48db-bbb5-f00522b53261\") " Jul 02 19:12:54.184859 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:54.184697 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqp7r\" (UniqueName: \"kubernetes.io/projected/c4b31346-488b-48db-bbb5-f00522b53261-kube-api-access-wqp7r\") pod \"c4b31346-488b-48db-bbb5-f00522b53261\" (UID: \"c4b31346-488b-48db-bbb5-f00522b53261\") " Jul 02 19:12:54.184859 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:54.184737 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4b31346-488b-48db-bbb5-f00522b53261-bundle\") pod \"c4b31346-488b-48db-bbb5-f00522b53261\" (UID: \"c4b31346-488b-48db-bbb5-f00522b53261\") " Jul 02 19:12:54.185584 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:54.185554 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b31346-488b-48db-bbb5-f00522b53261-bundle" (OuterVolumeSpecName: "bundle") pod "c4b31346-488b-48db-bbb5-f00522b53261" (UID: "c4b31346-488b-48db-bbb5-f00522b53261"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:12:54.186807 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:54.186785 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4b31346-488b-48db-bbb5-f00522b53261-kube-api-access-wqp7r" (OuterVolumeSpecName: "kube-api-access-wqp7r") pod "c4b31346-488b-48db-bbb5-f00522b53261" (UID: "c4b31346-488b-48db-bbb5-f00522b53261"). InnerVolumeSpecName "kube-api-access-wqp7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:12:54.190382 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:54.190345 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b31346-488b-48db-bbb5-f00522b53261-util" (OuterVolumeSpecName: "util") pod "c4b31346-488b-48db-bbb5-f00522b53261" (UID: "c4b31346-488b-48db-bbb5-f00522b53261"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:12:54.286016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:54.285938 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqp7r\" (UniqueName: \"kubernetes.io/projected/c4b31346-488b-48db-bbb5-f00522b53261-kube-api-access-wqp7r\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:12:54.286016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:54.285966 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4b31346-488b-48db-bbb5-f00522b53261-bundle\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:12:54.286016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:54.285975 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4b31346-488b-48db-bbb5-f00522b53261-util\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:12:54.997579 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:54.997540 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" event={"ID":"c4b31346-488b-48db-bbb5-f00522b53261","Type":"ContainerDied","Data":"254e7bb67ace49e39a02753ba845ea3924a2e5f21a463e9e59441591ff26dc09"} Jul 02 19:12:54.997751 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:54.997592 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="254e7bb67ace49e39a02753ba845ea3924a2e5f21a463e9e59441591ff26dc09" Jul 02 19:12:54.997751 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:12:54.997607 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl5sn8" Jul 02 19:13:03.410197 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.410163 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t"] Jul 02 19:13:03.410660 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.410450 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4b31346-488b-48db-bbb5-f00522b53261" containerName="util" Jul 02 19:13:03.410660 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.410462 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4b31346-488b-48db-bbb5-f00522b53261" containerName="util" Jul 02 19:13:03.410660 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.410470 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4b31346-488b-48db-bbb5-f00522b53261" containerName="extract" Jul 02 19:13:03.410660 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.410476 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4b31346-488b-48db-bbb5-f00522b53261" containerName="extract" Jul 02 19:13:03.410660 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.410484 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4b31346-488b-48db-bbb5-f00522b53261" containerName="pull" Jul 02 19:13:03.410660 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.410490 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4b31346-488b-48db-bbb5-f00522b53261" containerName="pull" Jul 02 19:13:03.410660 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.410542 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4b31346-488b-48db-bbb5-f00522b53261" containerName="extract" Jul 02 19:13:03.414959 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.414852 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:03.418146 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.418123 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zwcgm\"" Jul 02 19:13:03.418933 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.418914 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 02 19:13:03.419212 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.419194 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 02 19:13:03.419212 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.419204 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 02 19:13:03.419339 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.419246 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 02 19:13:03.450458 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.450424 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6439ecfe-740b-423e-b6d7-fd44ad306c87-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8ddf44b5-szh7t\" (UID: \"6439ecfe-740b-423e-b6d7-fd44ad306c87\") " pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:03.450630 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.450462 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6439ecfe-740b-423e-b6d7-fd44ad306c87-manifests\") pod \"opendatahub-operator-controller-manager-d8ddf44b5-szh7t\" (UID: \"6439ecfe-740b-423e-b6d7-fd44ad306c87\") " pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:03.450630 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.450538 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdc2f\" (UniqueName: \"kubernetes.io/projected/6439ecfe-740b-423e-b6d7-fd44ad306c87-kube-api-access-jdc2f\") pod \"opendatahub-operator-controller-manager-d8ddf44b5-szh7t\" (UID: \"6439ecfe-740b-423e-b6d7-fd44ad306c87\") " pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:03.450630 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.450588 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6439ecfe-740b-423e-b6d7-fd44ad306c87-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8ddf44b5-szh7t\" (UID: \"6439ecfe-740b-423e-b6d7-fd44ad306c87\") " pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:03.450787 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.450642 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6439ecfe-740b-423e-b6d7-fd44ad306c87-tmp\") pod \"opendatahub-operator-controller-manager-d8ddf44b5-szh7t\" (UID: \"6439ecfe-740b-423e-b6d7-fd44ad306c87\") " pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:03.454507 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.454483 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t"] Jul 02 19:13:03.551606 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.551570 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6439ecfe-740b-423e-b6d7-fd44ad306c87-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8ddf44b5-szh7t\" (UID: \"6439ecfe-740b-423e-b6d7-fd44ad306c87\") " pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:03.551606 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.551608 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6439ecfe-740b-423e-b6d7-fd44ad306c87-manifests\") pod \"opendatahub-operator-controller-manager-d8ddf44b5-szh7t\" (UID: \"6439ecfe-740b-423e-b6d7-fd44ad306c87\") " pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:03.551820 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.551628 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdc2f\" (UniqueName: \"kubernetes.io/projected/6439ecfe-740b-423e-b6d7-fd44ad306c87-kube-api-access-jdc2f\") pod \"opendatahub-operator-controller-manager-d8ddf44b5-szh7t\" (UID: \"6439ecfe-740b-423e-b6d7-fd44ad306c87\") " pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:03.551820 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.551653 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6439ecfe-740b-423e-b6d7-fd44ad306c87-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8ddf44b5-szh7t\" (UID: \"6439ecfe-740b-423e-b6d7-fd44ad306c87\") " pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:03.551820 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.551693 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6439ecfe-740b-423e-b6d7-fd44ad306c87-tmp\") pod \"opendatahub-operator-controller-manager-d8ddf44b5-szh7t\" (UID: \"6439ecfe-740b-423e-b6d7-fd44ad306c87\") " pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:03.552073 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.552049 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6439ecfe-740b-423e-b6d7-fd44ad306c87-manifests\") pod \"opendatahub-operator-controller-manager-d8ddf44b5-szh7t\" (UID: \"6439ecfe-740b-423e-b6d7-fd44ad306c87\") " pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:03.554108 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.554083 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6439ecfe-740b-423e-b6d7-fd44ad306c87-tmp\") pod \"opendatahub-operator-controller-manager-d8ddf44b5-szh7t\" (UID: \"6439ecfe-740b-423e-b6d7-fd44ad306c87\") " pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:03.554254 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.554236 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6439ecfe-740b-423e-b6d7-fd44ad306c87-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8ddf44b5-szh7t\" (UID: \"6439ecfe-740b-423e-b6d7-fd44ad306c87\") " pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:03.554294 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.554277 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6439ecfe-740b-423e-b6d7-fd44ad306c87-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8ddf44b5-szh7t\" (UID: \"6439ecfe-740b-423e-b6d7-fd44ad306c87\") " pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:03.572820 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.572785 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdc2f\" (UniqueName: \"kubernetes.io/projected/6439ecfe-740b-423e-b6d7-fd44ad306c87-kube-api-access-jdc2f\") pod \"opendatahub-operator-controller-manager-d8ddf44b5-szh7t\" (UID: \"6439ecfe-740b-423e-b6d7-fd44ad306c87\") " pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:03.725383 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.725310 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:03.858063 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:03.858036 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t"] Jul 02 19:13:03.860447 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:13:03.860414 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6439ecfe_740b_423e_b6d7_fd44ad306c87.slice/crio-d880e61d6027ce3a4c59a2306c9180852a310740fb9712f4a4b26084defa5a81 WatchSource:0}: Error finding container d880e61d6027ce3a4c59a2306c9180852a310740fb9712f4a4b26084defa5a81: Status 404 returned error can't find the container with id d880e61d6027ce3a4c59a2306c9180852a310740fb9712f4a4b26084defa5a81 Jul 02 19:13:04.035468 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:04.035369 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" event={"ID":"6439ecfe-740b-423e-b6d7-fd44ad306c87","Type":"ContainerStarted","Data":"d880e61d6027ce3a4c59a2306c9180852a310740fb9712f4a4b26084defa5a81"} Jul 02 19:13:07.051365 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:07.051333 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" event={"ID":"6439ecfe-740b-423e-b6d7-fd44ad306c87","Type":"ContainerStarted","Data":"15fcc870e658e8dfb69235362218183163f7f71ac09763319077427ca45f7d37"} Jul 02 19:13:08.056669 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:08.056639 2578 generic.go:356] "Generic (PLEG): container finished" podID="6439ecfe-740b-423e-b6d7-fd44ad306c87" containerID="15fcc870e658e8dfb69235362218183163f7f71ac09763319077427ca45f7d37" exitCode=0 Jul 02 19:13:08.056993 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:08.056724 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" event={"ID":"6439ecfe-740b-423e-b6d7-fd44ad306c87","Type":"ContainerDied","Data":"15fcc870e658e8dfb69235362218183163f7f71ac09763319077427ca45f7d37"} Jul 02 19:13:09.062600 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:09.062559 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" event={"ID":"6439ecfe-740b-423e-b6d7-fd44ad306c87","Type":"ContainerStarted","Data":"5741eb081723755f148ce073565b7c990bafaa7f0874a5c26b0208a31d43941b"} Jul 02 19:13:09.063047 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:09.062660 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:09.085659 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:09.085607 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" podStartSLOduration=1.473356908 podStartE2EDuration="6.085591386s" totalImagesPullingTime="3.547027068s" totalInitContainerRuntime="1.06520741s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:09.083983773 +0000 UTC m=+331.886799238" watchObservedRunningTime="2026-07-02 19:13:09.085591386 +0000 UTC m=+331.888406837" Jul 02 19:13:20.068607 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:20.068579 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d8ddf44b5-szh7t" Jul 02 19:13:23.381214 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:23.381169 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fw6d8"] Jul 02 19:13:23.389296 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:23.389275 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fw6d8" Jul 02 19:13:23.400576 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:23.400558 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 02 19:13:23.401316 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:23.401299 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-k8p5s\"" Jul 02 19:13:23.421067 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:23.421046 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fw6d8"] Jul 02 19:13:23.498044 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:23.498009 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6316e3d6-05d3-46e3-8fe2-986e631c898b-cert\") pod \"odh-model-controller-7998c67dc4-fw6d8\" (UID: \"6316e3d6-05d3-46e3-8fe2-986e631c898b\") " pod="opendatahub/odh-model-controller-7998c67dc4-fw6d8" Jul 02 19:13:23.498184 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:23.498067 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hn4s\" (UniqueName: \"kubernetes.io/projected/6316e3d6-05d3-46e3-8fe2-986e631c898b-kube-api-access-8hn4s\") pod \"odh-model-controller-7998c67dc4-fw6d8\" (UID: \"6316e3d6-05d3-46e3-8fe2-986e631c898b\") " pod="opendatahub/odh-model-controller-7998c67dc4-fw6d8" Jul 02 19:13:23.598686 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:23.598656 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6316e3d6-05d3-46e3-8fe2-986e631c898b-cert\") pod \"odh-model-controller-7998c67dc4-fw6d8\" (UID: \"6316e3d6-05d3-46e3-8fe2-986e631c898b\") " pod="opendatahub/odh-model-controller-7998c67dc4-fw6d8" Jul 02 19:13:23.598835 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:23.598706 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hn4s\" (UniqueName: \"kubernetes.io/projected/6316e3d6-05d3-46e3-8fe2-986e631c898b-kube-api-access-8hn4s\") pod \"odh-model-controller-7998c67dc4-fw6d8\" (UID: \"6316e3d6-05d3-46e3-8fe2-986e631c898b\") " pod="opendatahub/odh-model-controller-7998c67dc4-fw6d8" Jul 02 19:13:23.601183 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:23.601159 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6316e3d6-05d3-46e3-8fe2-986e631c898b-cert\") pod \"odh-model-controller-7998c67dc4-fw6d8\" (UID: \"6316e3d6-05d3-46e3-8fe2-986e631c898b\") " pod="opendatahub/odh-model-controller-7998c67dc4-fw6d8" Jul 02 19:13:23.610514 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:23.610493 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hn4s\" (UniqueName: \"kubernetes.io/projected/6316e3d6-05d3-46e3-8fe2-986e631c898b-kube-api-access-8hn4s\") pod \"odh-model-controller-7998c67dc4-fw6d8\" (UID: \"6316e3d6-05d3-46e3-8fe2-986e631c898b\") " pod="opendatahub/odh-model-controller-7998c67dc4-fw6d8" Jul 02 19:13:23.698726 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:23.698636 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fw6d8" Jul 02 19:13:23.829282 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:23.829252 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fw6d8"] Jul 02 19:13:23.832156 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:13:23.832126 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6316e3d6_05d3_46e3_8fe2_986e631c898b.slice/crio-1436bbe63ab34d72718a3a010310ee8927259105bf9298ef6c9231e7e7ef5506 WatchSource:0}: Error finding container 1436bbe63ab34d72718a3a010310ee8927259105bf9298ef6c9231e7e7ef5506: Status 404 returned error can't find the container with id 1436bbe63ab34d72718a3a010310ee8927259105bf9298ef6c9231e7e7ef5506 Jul 02 19:13:24.121447 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:24.121416 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fw6d8" event={"ID":"6316e3d6-05d3-46e3-8fe2-986e631c898b","Type":"ContainerStarted","Data":"1436bbe63ab34d72718a3a010310ee8927259105bf9298ef6c9231e7e7ef5506"} Jul 02 19:13:24.222740 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:24.222709 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rf6t5"] Jul 02 19:13:24.227208 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:24.227193 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rf6t5" Jul 02 19:13:24.229795 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:24.229775 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 02 19:13:24.229881 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:24.229777 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-kgfmm\"" Jul 02 19:13:24.238056 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:24.238037 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rf6t5"] Jul 02 19:13:24.305374 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:24.305353 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1931d911-68dd-48ac-836d-b57761b26166-tls-certs\") pod \"model-serving-api-54d89bd5fc-rf6t5\" (UID: \"1931d911-68dd-48ac-836d-b57761b26166\") " pod="opendatahub/model-serving-api-54d89bd5fc-rf6t5" Jul 02 19:13:24.305505 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:24.305435 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch9vv\" (UniqueName: \"kubernetes.io/projected/1931d911-68dd-48ac-836d-b57761b26166-kube-api-access-ch9vv\") pod \"model-serving-api-54d89bd5fc-rf6t5\" (UID: \"1931d911-68dd-48ac-836d-b57761b26166\") " pod="opendatahub/model-serving-api-54d89bd5fc-rf6t5" Jul 02 19:13:24.406702 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:24.406630 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch9vv\" (UniqueName: \"kubernetes.io/projected/1931d911-68dd-48ac-836d-b57761b26166-kube-api-access-ch9vv\") pod \"model-serving-api-54d89bd5fc-rf6t5\" (UID: \"1931d911-68dd-48ac-836d-b57761b26166\") " pod="opendatahub/model-serving-api-54d89bd5fc-rf6t5" Jul 02 19:13:24.406702 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:24.406687 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1931d911-68dd-48ac-836d-b57761b26166-tls-certs\") pod \"model-serving-api-54d89bd5fc-rf6t5\" (UID: \"1931d911-68dd-48ac-836d-b57761b26166\") " pod="opendatahub/model-serving-api-54d89bd5fc-rf6t5" Jul 02 19:13:24.407214 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:13:24.406789 2578 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 02 19:13:24.407214 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:13:24.406849 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1931d911-68dd-48ac-836d-b57761b26166-tls-certs podName:1931d911-68dd-48ac-836d-b57761b26166 nodeName:}" failed. No retries permitted until 2026-07-02 19:13:24.90682731 +0000 UTC m=+347.709642741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/1931d911-68dd-48ac-836d-b57761b26166-tls-certs") pod "model-serving-api-54d89bd5fc-rf6t5" (UID: "1931d911-68dd-48ac-836d-b57761b26166") : secret "model-serving-api-tls" not found Jul 02 19:13:24.415676 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:24.415650 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch9vv\" (UniqueName: \"kubernetes.io/projected/1931d911-68dd-48ac-836d-b57761b26166-kube-api-access-ch9vv\") pod \"model-serving-api-54d89bd5fc-rf6t5\" (UID: \"1931d911-68dd-48ac-836d-b57761b26166\") " pod="opendatahub/model-serving-api-54d89bd5fc-rf6t5" Jul 02 19:13:24.911518 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:24.911484 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1931d911-68dd-48ac-836d-b57761b26166-tls-certs\") pod \"model-serving-api-54d89bd5fc-rf6t5\" (UID: \"1931d911-68dd-48ac-836d-b57761b26166\") " pod="opendatahub/model-serving-api-54d89bd5fc-rf6t5" Jul 02 19:13:24.914558 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:24.914531 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1931d911-68dd-48ac-836d-b57761b26166-tls-certs\") pod \"model-serving-api-54d89bd5fc-rf6t5\" (UID: \"1931d911-68dd-48ac-836d-b57761b26166\") " pod="opendatahub/model-serving-api-54d89bd5fc-rf6t5" Jul 02 19:13:25.137318 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:25.137284 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rf6t5" Jul 02 19:13:25.756959 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:25.756926 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rf6t5"] Jul 02 19:13:26.068619 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:26.068593 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7bdf57ddb6-4v27p"] Jul 02 19:13:26.072203 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:26.072182 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bdf57ddb6-4v27p" Jul 02 19:13:26.074361 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:26.074337 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 02 19:13:26.074590 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:26.074570 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 02 19:13:26.074735 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:26.074707 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-44hnj\"" Jul 02 19:13:26.083558 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:26.083536 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bdf57ddb6-4v27p"] Jul 02 19:13:26.222261 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:26.222228 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c0bbba4-27e1-4715-80d5-e5e157e52e7c-tls-certs\") pod \"kube-auth-proxy-7bdf57ddb6-4v27p\" (UID: \"1c0bbba4-27e1-4715-80d5-e5e157e52e7c\") " pod="openshift-ingress/kube-auth-proxy-7bdf57ddb6-4v27p" Jul 02 19:13:26.222459 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:26.222298 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85gjz\" (UniqueName: \"kubernetes.io/projected/1c0bbba4-27e1-4715-80d5-e5e157e52e7c-kube-api-access-85gjz\") pod \"kube-auth-proxy-7bdf57ddb6-4v27p\" (UID: \"1c0bbba4-27e1-4715-80d5-e5e157e52e7c\") " pod="openshift-ingress/kube-auth-proxy-7bdf57ddb6-4v27p" Jul 02 19:13:26.222459 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:26.222361 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c0bbba4-27e1-4715-80d5-e5e157e52e7c-tmp\") pod \"kube-auth-proxy-7bdf57ddb6-4v27p\" (UID: \"1c0bbba4-27e1-4715-80d5-e5e157e52e7c\") " pod="openshift-ingress/kube-auth-proxy-7bdf57ddb6-4v27p" Jul 02 19:13:26.235896 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:13:26.235865 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1931d911_68dd_48ac_836d_b57761b26166.slice/crio-99ac28950ad7ca523bd24819716db949bd2ea021167065f93ad7adb8447499a0 WatchSource:0}: Error finding container 99ac28950ad7ca523bd24819716db949bd2ea021167065f93ad7adb8447499a0: Status 404 returned error can't find the container with id 99ac28950ad7ca523bd24819716db949bd2ea021167065f93ad7adb8447499a0 Jul 02 19:13:26.323028 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:26.322995 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85gjz\" (UniqueName: \"kubernetes.io/projected/1c0bbba4-27e1-4715-80d5-e5e157e52e7c-kube-api-access-85gjz\") pod \"kube-auth-proxy-7bdf57ddb6-4v27p\" (UID: \"1c0bbba4-27e1-4715-80d5-e5e157e52e7c\") " pod="openshift-ingress/kube-auth-proxy-7bdf57ddb6-4v27p" Jul 02 19:13:26.323135 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:26.323063 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c0bbba4-27e1-4715-80d5-e5e157e52e7c-tmp\") pod \"kube-auth-proxy-7bdf57ddb6-4v27p\" (UID: \"1c0bbba4-27e1-4715-80d5-e5e157e52e7c\") " pod="openshift-ingress/kube-auth-proxy-7bdf57ddb6-4v27p" Jul 02 19:13:26.323135 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:26.323113 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c0bbba4-27e1-4715-80d5-e5e157e52e7c-tls-certs\") pod \"kube-auth-proxy-7bdf57ddb6-4v27p\" (UID: \"1c0bbba4-27e1-4715-80d5-e5e157e52e7c\") " pod="openshift-ingress/kube-auth-proxy-7bdf57ddb6-4v27p" Jul 02 19:13:26.325149 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:26.325129 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c0bbba4-27e1-4715-80d5-e5e157e52e7c-tmp\") pod \"kube-auth-proxy-7bdf57ddb6-4v27p\" (UID: \"1c0bbba4-27e1-4715-80d5-e5e157e52e7c\") " pod="openshift-ingress/kube-auth-proxy-7bdf57ddb6-4v27p" Jul 02 19:13:26.325505 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:26.325485 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c0bbba4-27e1-4715-80d5-e5e157e52e7c-tls-certs\") pod \"kube-auth-proxy-7bdf57ddb6-4v27p\" (UID: \"1c0bbba4-27e1-4715-80d5-e5e157e52e7c\") " pod="openshift-ingress/kube-auth-proxy-7bdf57ddb6-4v27p" Jul 02 19:13:26.331171 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:26.331141 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85gjz\" (UniqueName: \"kubernetes.io/projected/1c0bbba4-27e1-4715-80d5-e5e157e52e7c-kube-api-access-85gjz\") pod \"kube-auth-proxy-7bdf57ddb6-4v27p\" (UID: \"1c0bbba4-27e1-4715-80d5-e5e157e52e7c\") " pod="openshift-ingress/kube-auth-proxy-7bdf57ddb6-4v27p" Jul 02 19:13:26.384297 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:26.384263 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bdf57ddb6-4v27p" Jul 02 19:13:26.509097 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:26.509072 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bdf57ddb6-4v27p"] Jul 02 19:13:26.511192 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:13:26.511167 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c0bbba4_27e1_4715_80d5_e5e157e52e7c.slice/crio-2c2e9f45c68249dbc14aaadd5cf48ab617b8966b94c45b9abf2e3d9e16646611 WatchSource:0}: Error finding container 2c2e9f45c68249dbc14aaadd5cf48ab617b8966b94c45b9abf2e3d9e16646611: Status 404 returned error can't find the container with id 2c2e9f45c68249dbc14aaadd5cf48ab617b8966b94c45b9abf2e3d9e16646611 Jul 02 19:13:27.138370 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.138328 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bdf57ddb6-4v27p" event={"ID":"1c0bbba4-27e1-4715-80d5-e5e157e52e7c","Type":"ContainerStarted","Data":"2c2e9f45c68249dbc14aaadd5cf48ab617b8966b94c45b9abf2e3d9e16646611"} Jul 02 19:13:27.140104 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.140071 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fw6d8" event={"ID":"6316e3d6-05d3-46e3-8fe2-986e631c898b","Type":"ContainerStarted","Data":"6e0903a525ccee1349761560fe39e2b90b86a068c902700f6f75cda6c8f4b7f1"} Jul 02 19:13:27.140271 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.140180 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-fw6d8" Jul 02 19:13:27.141810 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.141785 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rf6t5" event={"ID":"1931d911-68dd-48ac-836d-b57761b26166","Type":"ContainerStarted","Data":"99ac28950ad7ca523bd24819716db949bd2ea021167065f93ad7adb8447499a0"} Jul 02 19:13:27.162045 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.161994 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-fw6d8" podStartSLOduration=1.702440207 podStartE2EDuration="4.161980824s" totalImagesPullingTime="2.459540617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:27.161238303 +0000 UTC m=+349.964053748" watchObservedRunningTime="2026-07-02 19:13:27.161980824 +0000 UTC m=+349.964796297" Jul 02 19:13:27.180298 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.180264 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-h94ld"] Jul 02 19:13:27.185785 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.185635 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.188709 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.188690 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 02 19:13:27.188855 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.188830 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 02 19:13:27.188991 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.188974 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-j7xml\"" Jul 02 19:13:27.189055 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.189015 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 02 19:13:27.189282 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.189268 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 02 19:13:27.198102 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.198083 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-h94ld"] Jul 02 19:13:27.332265 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.332226 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fa17560c-0240-48c0-8889-b667b60383d6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.332435 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.332283 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa17560c-0240-48c0-8889-b667b60383d6-local-certs\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.332435 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.332339 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa17560c-0240-48c0-8889-b667b60383d6-istio-token\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.332435 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.332369 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.332435 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.332414 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59l62\" (UniqueName: \"kubernetes.io/projected/fa17560c-0240-48c0-8889-b667b60383d6-kube-api-access-59l62\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.332656 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.332453 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-cacerts\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.332656 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.332490 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.433329 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.433243 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.433494 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.433335 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fa17560c-0240-48c0-8889-b667b60383d6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.433494 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.433379 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa17560c-0240-48c0-8889-b667b60383d6-local-certs\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.433494 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.433418 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa17560c-0240-48c0-8889-b667b60383d6-istio-token\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.433494 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.433444 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.433494 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.433464 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59l62\" (UniqueName: \"kubernetes.io/projected/fa17560c-0240-48c0-8889-b667b60383d6-kube-api-access-59l62\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.433494 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.433494 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-cacerts\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.434585 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.434552 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fa17560c-0240-48c0-8889-b667b60383d6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.436277 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.436231 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.436414 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.436344 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.436839 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.436817 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa17560c-0240-48c0-8889-b667b60383d6-local-certs\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.437119 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.437094 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-cacerts\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.442591 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.442565 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa17560c-0240-48c0-8889-b667b60383d6-istio-token\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.442699 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.442632 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59l62\" (UniqueName: \"kubernetes.io/projected/fa17560c-0240-48c0-8889-b667b60383d6-kube-api-access-59l62\") pod \"istiod-openshift-gateway-7574459656-h94ld\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.499517 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.499491 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:27.654219 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:27.654196 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-h94ld"] Jul 02 19:13:27.656295 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:13:27.656264 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa17560c_0240_48c0_8889_b667b60383d6.slice/crio-b62486dc1ec6d7281c2f4b8e63bf629bcb124e51d314bc00c3f0b23a2a6d6e76 WatchSource:0}: Error finding container b62486dc1ec6d7281c2f4b8e63bf629bcb124e51d314bc00c3f0b23a2a6d6e76: Status 404 returned error can't find the container with id b62486dc1ec6d7281c2f4b8e63bf629bcb124e51d314bc00c3f0b23a2a6d6e76 Jul 02 19:13:28.146886 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:28.146845 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" event={"ID":"fa17560c-0240-48c0-8889-b667b60383d6","Type":"ContainerStarted","Data":"b62486dc1ec6d7281c2f4b8e63bf629bcb124e51d314bc00c3f0b23a2a6d6e76"} Jul 02 19:13:29.154274 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:29.154216 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rf6t5" event={"ID":"1931d911-68dd-48ac-836d-b57761b26166","Type":"ContainerStarted","Data":"da5e0d756188fd50e76bad572fce85ad0c801bf2eb3ae4c438de75e1cf6680a7"} Jul 02 19:13:29.154810 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:29.154370 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-rf6t5" Jul 02 19:13:29.171800 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:29.171749 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-rf6t5" podStartSLOduration=3.230167459 podStartE2EDuration="5.171730994s" totalImagesPullingTime="1.941563535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:29.171289876 +0000 UTC m=+351.974105351" watchObservedRunningTime="2026-07-02 19:13:29.171730994 +0000 UTC m=+351.974546446" Jul 02 19:13:30.142498 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:30.142425 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:13:30.142614 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:30.142545 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:13:31.167735 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:31.167694 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bdf57ddb6-4v27p" event={"ID":"1c0bbba4-27e1-4715-80d5-e5e157e52e7c","Type":"ContainerStarted","Data":"6ec06e72214dafe5ab067f88feec39dc4511c0e36d0294d87306c673cc2ed855"} Jul 02 19:13:31.169549 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:31.169523 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" event={"ID":"fa17560c-0240-48c0-8889-b667b60383d6","Type":"ContainerStarted","Data":"30b72589f1d4dd0d84ee6ba46f994f50c42544d928c155d6d4ba9abb8ab73f18"} Jul 02 19:13:31.169786 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:31.169770 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:31.171317 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:31.171295 2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-h94ld container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 02 19:13:31.171429 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:31.171353 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" podUID="fa17560c-0240-48c0-8889-b667b60383d6" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 19:13:31.184107 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:31.184064 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7bdf57ddb6-4v27p" podStartSLOduration=0.694293389 podStartE2EDuration="5.184051497s" totalImagesPullingTime="4.489758108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:31.183598191 +0000 UTC m=+353.986413661" watchObservedRunningTime="2026-07-02 19:13:31.184051497 +0000 UTC m=+353.986866948" Jul 02 19:13:31.202502 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:31.202465 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" podStartSLOduration=1.719120775 podStartE2EDuration="4.202451391s" totalImagesPullingTime="2.483330616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:31.201922108 +0000 UTC m=+354.004737572" watchObservedRunningTime="2026-07-02 19:13:31.202451391 +0000 UTC m=+354.005266842" Jul 02 19:13:32.154924 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.154893 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt"] Jul 02 19:13:32.157682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.157666 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.161220 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.161189 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-cz99g\"" Jul 02 19:13:32.175019 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.174995 2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-h94ld container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 02 19:13:32.175284 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.175053 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" podUID="fa17560c-0240-48c0-8889-b667b60383d6" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 19:13:32.178719 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.178700 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt"] Jul 02 19:13:32.274763 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.274733 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ee861e5-4888-4347-9b0f-65974149e684-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.274884 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.274764 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ee861e5-4888-4347-9b0f-65974149e684-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.274884 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.274794 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrnm7\" (UniqueName: \"kubernetes.io/projected/2ee861e5-4888-4347-9b0f-65974149e684-kube-api-access-lrnm7\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.275046 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.275023 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ee861e5-4888-4347-9b0f-65974149e684-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.275105 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.275072 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ee861e5-4888-4347-9b0f-65974149e684-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.275154 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.275123 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ee861e5-4888-4347-9b0f-65974149e684-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.275333 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.275310 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ee861e5-4888-4347-9b0f-65974149e684-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.275374 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.275351 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ee861e5-4888-4347-9b0f-65974149e684-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.275443 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.275387 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ee861e5-4888-4347-9b0f-65974149e684-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.376380 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.376353 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ee861e5-4888-4347-9b0f-65974149e684-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.376496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.376384 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ee861e5-4888-4347-9b0f-65974149e684-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.376496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.376432 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrnm7\" (UniqueName: \"kubernetes.io/projected/2ee861e5-4888-4347-9b0f-65974149e684-kube-api-access-lrnm7\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.376496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.376476 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ee861e5-4888-4347-9b0f-65974149e684-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.376641 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.376508 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ee861e5-4888-4347-9b0f-65974149e684-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.376641 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.376538 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ee861e5-4888-4347-9b0f-65974149e684-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.376641 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.376574 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ee861e5-4888-4347-9b0f-65974149e684-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.376641 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.376602 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ee861e5-4888-4347-9b0f-65974149e684-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.376641 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.376634 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ee861e5-4888-4347-9b0f-65974149e684-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.376873 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.376845 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ee861e5-4888-4347-9b0f-65974149e684-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.376993 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.376974 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ee861e5-4888-4347-9b0f-65974149e684-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.377079 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.377057 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ee861e5-4888-4347-9b0f-65974149e684-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.377132 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.377116 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ee861e5-4888-4347-9b0f-65974149e684-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.377174 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.377145 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ee861e5-4888-4347-9b0f-65974149e684-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.378779 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.378760 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ee861e5-4888-4347-9b0f-65974149e684-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.379015 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.378992 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ee861e5-4888-4347-9b0f-65974149e684-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.393795 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.393770 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ee861e5-4888-4347-9b0f-65974149e684-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.399527 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.399504 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrnm7\" (UniqueName: \"kubernetes.io/projected/2ee861e5-4888-4347-9b0f-65974149e684-kube-api-access-lrnm7\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wgddt\" (UID: \"2ee861e5-4888-4347-9b0f-65974149e684\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.467738 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.467682 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:32.608482 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:13:32.608449 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ee861e5_4888_4347_9b0f_65974149e684.slice/crio-8e004a0836eed8c641c1d966ba726d097d106fa07e3a81a8bee06485c07d21a4 WatchSource:0}: Error finding container 8e004a0836eed8c641c1d966ba726d097d106fa07e3a81a8bee06485c07d21a4: Status 404 returned error can't find the container with id 8e004a0836eed8c641c1d966ba726d097d106fa07e3a81a8bee06485c07d21a4 Jul 02 19:13:32.608628 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:32.608486 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt"] Jul 02 19:13:33.179410 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:33.179366 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" event={"ID":"2ee861e5-4888-4347-9b0f-65974149e684","Type":"ContainerStarted","Data":"8e004a0836eed8c641c1d966ba726d097d106fa07e3a81a8bee06485c07d21a4"} Jul 02 19:13:33.929703 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:33.929671 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qhzkx"] Jul 02 19:13:33.933262 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:33.933236 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kzj2c"] Jul 02 19:13:33.933420 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:33.933371 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qhzkx" Jul 02 19:13:33.936351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:33.936332 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzj2c" Jul 02 19:13:33.952765 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:33.952739 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-mc8xb\"" Jul 02 19:13:33.952979 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:33.952959 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 02 19:13:33.953120 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:33.953087 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kzj2c"] Jul 02 19:13:33.956002 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:33.955985 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-mkltm\"" Jul 02 19:13:33.966792 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:33.966770 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 02 19:13:33.978960 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:33.978938 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qhzkx"] Jul 02 19:13:34.089940 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:34.089904 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65ztc\" (UniqueName: \"kubernetes.io/projected/b175954f-c460-4009-8bb7-40d192370efb-kube-api-access-65ztc\") pod \"llmisvc-controller-manager-777c6bd7d5-qhzkx\" (UID: \"b175954f-c460-4009-8bb7-40d192370efb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qhzkx" Jul 02 19:13:34.089940 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:34.089942 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b175954f-c460-4009-8bb7-40d192370efb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qhzkx\" (UID: \"b175954f-c460-4009-8bb7-40d192370efb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qhzkx" Jul 02 19:13:34.090130 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:34.089963 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqmbr\" (UniqueName: \"kubernetes.io/projected/66139e53-0f64-4c2a-9fb1-bbf4b6267f35-kube-api-access-fqmbr\") pod \"kserve-controller-manager-d6f4b65b8-kzj2c\" (UID: \"66139e53-0f64-4c2a-9fb1-bbf4b6267f35\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzj2c" Jul 02 19:13:34.090130 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:34.089984 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66139e53-0f64-4c2a-9fb1-bbf4b6267f35-cert\") pod \"kserve-controller-manager-d6f4b65b8-kzj2c\" (UID: \"66139e53-0f64-4c2a-9fb1-bbf4b6267f35\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzj2c" Jul 02 19:13:34.190604 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:34.190529 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65ztc\" (UniqueName: \"kubernetes.io/projected/b175954f-c460-4009-8bb7-40d192370efb-kube-api-access-65ztc\") pod \"llmisvc-controller-manager-777c6bd7d5-qhzkx\" (UID: \"b175954f-c460-4009-8bb7-40d192370efb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qhzkx" Jul 02 19:13:34.190604 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:34.190570 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b175954f-c460-4009-8bb7-40d192370efb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qhzkx\" (UID: \"b175954f-c460-4009-8bb7-40d192370efb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qhzkx" Jul 02 19:13:34.190604 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:34.190602 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqmbr\" (UniqueName: \"kubernetes.io/projected/66139e53-0f64-4c2a-9fb1-bbf4b6267f35-kube-api-access-fqmbr\") pod \"kserve-controller-manager-d6f4b65b8-kzj2c\" (UID: \"66139e53-0f64-4c2a-9fb1-bbf4b6267f35\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzj2c" Jul 02 19:13:34.191152 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:34.190626 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66139e53-0f64-4c2a-9fb1-bbf4b6267f35-cert\") pod \"kserve-controller-manager-d6f4b65b8-kzj2c\" (UID: \"66139e53-0f64-4c2a-9fb1-bbf4b6267f35\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzj2c" Jul 02 19:13:34.193310 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:34.193284 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66139e53-0f64-4c2a-9fb1-bbf4b6267f35-cert\") pod \"kserve-controller-manager-d6f4b65b8-kzj2c\" (UID: \"66139e53-0f64-4c2a-9fb1-bbf4b6267f35\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzj2c" Jul 02 19:13:34.193442 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:34.193373 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b175954f-c460-4009-8bb7-40d192370efb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qhzkx\" (UID: \"b175954f-c460-4009-8bb7-40d192370efb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qhzkx" Jul 02 19:13:34.208977 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:34.208951 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqmbr\" (UniqueName: \"kubernetes.io/projected/66139e53-0f64-4c2a-9fb1-bbf4b6267f35-kube-api-access-fqmbr\") pod \"kserve-controller-manager-d6f4b65b8-kzj2c\" (UID: \"66139e53-0f64-4c2a-9fb1-bbf4b6267f35\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzj2c" Jul 02 19:13:34.222909 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:34.222882 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65ztc\" (UniqueName: \"kubernetes.io/projected/b175954f-c460-4009-8bb7-40d192370efb-kube-api-access-65ztc\") pod \"llmisvc-controller-manager-777c6bd7d5-qhzkx\" (UID: \"b175954f-c460-4009-8bb7-40d192370efb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qhzkx" Jul 02 19:13:34.246616 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:34.246591 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qhzkx" Jul 02 19:13:34.254054 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:34.254037 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzj2c" Jul 02 19:13:35.164142 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:35.163895 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:13:35.164142 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:35.164029 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:13:35.164142 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:35.164078 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:13:35.175346 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:35.175326 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:35.265783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:35.265702 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qhzkx"] Jul 02 19:13:35.295494 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:35.295468 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kzj2c"] Jul 02 19:13:35.297200 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:13:35.297163 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66139e53_0f64_4c2a_9fb1_bbf4b6267f35.slice/crio-86115e3b06bbee08ea1251e835652eb8249a565f6aa97bc31000b275d72804bb WatchSource:0}: Error finding container 86115e3b06bbee08ea1251e835652eb8249a565f6aa97bc31000b275d72804bb: Status 404 returned error can't find the container with id 86115e3b06bbee08ea1251e835652eb8249a565f6aa97bc31000b275d72804bb Jul 02 19:13:36.203096 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:36.203056 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzj2c" event={"ID":"66139e53-0f64-4c2a-9fb1-bbf4b6267f35","Type":"ContainerStarted","Data":"86115e3b06bbee08ea1251e835652eb8249a565f6aa97bc31000b275d72804bb"} Jul 02 19:13:36.204722 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:36.204691 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qhzkx" event={"ID":"b175954f-c460-4009-8bb7-40d192370efb","Type":"ContainerStarted","Data":"6260d500539c8bef3ea374a4893d40fb475b118abbf4ac31d3c6dff1b7e9f687"} Jul 02 19:13:36.206445 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:36.206374 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" event={"ID":"2ee861e5-4888-4347-9b0f-65974149e684","Type":"ContainerStarted","Data":"3a7361e3fe9a2db75b465b4a389c76da42fadc315206d7f7c240e93f5cb4daae"} Jul 02 19:13:36.257094 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:36.256951 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" podStartSLOduration=1.703702688 podStartE2EDuration="4.256931664s" totalImagesPullingTime="2.553228976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:36.254699338 +0000 UTC m=+359.057514790" watchObservedRunningTime="2026-07-02 19:13:36.256931664 +0000 UTC m=+359.059747116" Jul 02 19:13:36.468991 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:36.468875 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:36.474476 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:36.474230 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:37.211480 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:37.211447 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:37.212437 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:37.212388 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wgddt" Jul 02 19:13:38.149308 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.149276 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-fw6d8" Jul 02 19:13:38.394472 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.394438 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-h94ld"] Jul 02 19:13:38.394668 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.394644 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" podUID="fa17560c-0240-48c0-8889-b667b60383d6" containerName="discovery" containerID="cri-o://30b72589f1d4dd0d84ee6ba46f994f50c42544d928c155d6d4ba9abb8ab73f18" gracePeriod=30 Jul 02 19:13:38.642483 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.642459 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:38.731941 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.731847 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-istio-csr-dns-cert\") pod \"fa17560c-0240-48c0-8889-b667b60383d6\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " Jul 02 19:13:38.731941 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.731928 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa17560c-0240-48c0-8889-b667b60383d6-istio-token\") pod \"fa17560c-0240-48c0-8889-b667b60383d6\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " Jul 02 19:13:38.732153 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.731951 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-cacerts\") pod \"fa17560c-0240-48c0-8889-b667b60383d6\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " Jul 02 19:13:38.732153 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.731971 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fa17560c-0240-48c0-8889-b667b60383d6-istio-csr-ca-configmap\") pod \"fa17560c-0240-48c0-8889-b667b60383d6\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " Jul 02 19:13:38.732153 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.731994 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59l62\" (UniqueName: \"kubernetes.io/projected/fa17560c-0240-48c0-8889-b667b60383d6-kube-api-access-59l62\") pod \"fa17560c-0240-48c0-8889-b667b60383d6\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " Jul 02 19:13:38.732153 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.732032 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-istio-kubeconfig\") pod \"fa17560c-0240-48c0-8889-b667b60383d6\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " Jul 02 19:13:38.732153 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.732064 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa17560c-0240-48c0-8889-b667b60383d6-local-certs\") pod \"fa17560c-0240-48c0-8889-b667b60383d6\" (UID: \"fa17560c-0240-48c0-8889-b667b60383d6\") " Jul 02 19:13:38.732731 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.732659 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa17560c-0240-48c0-8889-b667b60383d6-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "fa17560c-0240-48c0-8889-b667b60383d6" (UID: "fa17560c-0240-48c0-8889-b667b60383d6"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:13:38.736651 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.736609 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa17560c-0240-48c0-8889-b667b60383d6-istio-token" (OuterVolumeSpecName: "istio-token") pod "fa17560c-0240-48c0-8889-b667b60383d6" (UID: "fa17560c-0240-48c0-8889-b667b60383d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:13:38.736966 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.736835 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa17560c-0240-48c0-8889-b667b60383d6-kube-api-access-59l62" (OuterVolumeSpecName: "kube-api-access-59l62") pod "fa17560c-0240-48c0-8889-b667b60383d6" (UID: "fa17560c-0240-48c0-8889-b667b60383d6"). InnerVolumeSpecName "kube-api-access-59l62". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:13:38.736966 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.736886 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa17560c-0240-48c0-8889-b667b60383d6-local-certs" (OuterVolumeSpecName: "local-certs") pod "fa17560c-0240-48c0-8889-b667b60383d6" (UID: "fa17560c-0240-48c0-8889-b667b60383d6"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:13:38.736966 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.736938 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "fa17560c-0240-48c0-8889-b667b60383d6" (UID: "fa17560c-0240-48c0-8889-b667b60383d6"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:13:38.737138 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.736990 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "fa17560c-0240-48c0-8889-b667b60383d6" (UID: "fa17560c-0240-48c0-8889-b667b60383d6"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:13:38.737299 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.737268 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-cacerts" (OuterVolumeSpecName: "cacerts") pod "fa17560c-0240-48c0-8889-b667b60383d6" (UID: "fa17560c-0240-48c0-8889-b667b60383d6"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:13:38.833049 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.833020 2578 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-istio-kubeconfig\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:13:38.833049 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.833046 2578 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa17560c-0240-48c0-8889-b667b60383d6-local-certs\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:13:38.833049 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.833055 2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-istio-csr-dns-cert\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:13:38.833309 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.833065 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa17560c-0240-48c0-8889-b667b60383d6-istio-token\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:13:38.833309 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.833075 2578 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fa17560c-0240-48c0-8889-b667b60383d6-cacerts\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:13:38.833309 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.833082 2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fa17560c-0240-48c0-8889-b667b60383d6-istio-csr-ca-configmap\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:13:38.833309 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:38.833091 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59l62\" (UniqueName: \"kubernetes.io/projected/fa17560c-0240-48c0-8889-b667b60383d6-kube-api-access-59l62\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:13:39.223595 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:39.223561 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qhzkx" event={"ID":"b175954f-c460-4009-8bb7-40d192370efb","Type":"ContainerStarted","Data":"53cd5eb58533134c9feeddae985566202baffd1354e7e4eff6053b285b85b8e1"} Jul 02 19:13:39.223999 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:39.223655 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qhzkx" Jul 02 19:13:39.224862 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:39.224841 2578 generic.go:356] "Generic (PLEG): container finished" podID="fa17560c-0240-48c0-8889-b667b60383d6" containerID="30b72589f1d4dd0d84ee6ba46f994f50c42544d928c155d6d4ba9abb8ab73f18" exitCode=0 Jul 02 19:13:39.224939 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:39.224877 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" Jul 02 19:13:39.224984 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:39.224927 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" event={"ID":"fa17560c-0240-48c0-8889-b667b60383d6","Type":"ContainerDied","Data":"30b72589f1d4dd0d84ee6ba46f994f50c42544d928c155d6d4ba9abb8ab73f18"} Jul 02 19:13:39.224984 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:39.224978 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-h94ld" event={"ID":"fa17560c-0240-48c0-8889-b667b60383d6","Type":"ContainerDied","Data":"b62486dc1ec6d7281c2f4b8e63bf629bcb124e51d314bc00c3f0b23a2a6d6e76"} Jul 02 19:13:39.225063 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:39.225009 2578 scope.go:117] "RemoveContainer" containerID="30b72589f1d4dd0d84ee6ba46f994f50c42544d928c155d6d4ba9abb8ab73f18" Jul 02 19:13:39.226551 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:39.226527 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzj2c" event={"ID":"66139e53-0f64-4c2a-9fb1-bbf4b6267f35","Type":"ContainerStarted","Data":"17715602b9a3bc52b84a353bcdce55eebc370e8bfbfb5be0cccddb778e534010"} Jul 02 19:13:39.233643 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:39.233625 2578 scope.go:117] "RemoveContainer" containerID="30b72589f1d4dd0d84ee6ba46f994f50c42544d928c155d6d4ba9abb8ab73f18" Jul 02 19:13:39.233884 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:13:39.233866 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30b72589f1d4dd0d84ee6ba46f994f50c42544d928c155d6d4ba9abb8ab73f18\": container with ID starting with 30b72589f1d4dd0d84ee6ba46f994f50c42544d928c155d6d4ba9abb8ab73f18 not found: ID does not exist" containerID="30b72589f1d4dd0d84ee6ba46f994f50c42544d928c155d6d4ba9abb8ab73f18" Jul 02 19:13:39.233949 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:39.233892 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30b72589f1d4dd0d84ee6ba46f994f50c42544d928c155d6d4ba9abb8ab73f18"} err="failed to get container status \"30b72589f1d4dd0d84ee6ba46f994f50c42544d928c155d6d4ba9abb8ab73f18\": rpc error: code = NotFound desc = could not find container \"30b72589f1d4dd0d84ee6ba46f994f50c42544d928c155d6d4ba9abb8ab73f18\": container with ID starting with 30b72589f1d4dd0d84ee6ba46f994f50c42544d928c155d6d4ba9abb8ab73f18 not found: ID does not exist" Jul 02 19:13:39.244843 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:39.244802 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qhzkx" podStartSLOduration=3.082417148 podStartE2EDuration="6.244790783s" totalImagesPullingTime="3.162373635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:39.243103635 +0000 UTC m=+362.045919086" watchObservedRunningTime="2026-07-02 19:13:39.244790783 +0000 UTC m=+362.047606234" Jul 02 19:13:39.261804 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:39.261767 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzj2c" podStartSLOduration=3.1310458150000002 podStartE2EDuration="6.261754844s" totalImagesPullingTime="3.130709029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:39.260704864 +0000 UTC m=+362.063520315" watchObservedRunningTime="2026-07-02 19:13:39.261754844 +0000 UTC m=+362.064570296" Jul 02 19:13:39.277366 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:39.277346 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-h94ld"] Jul 02 19:13:39.280600 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:39.280579 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-h94ld"] Jul 02 19:13:39.731517 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:39.731484 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa17560c-0240-48c0-8889-b667b60383d6" path="/var/lib/kubelet/pods/fa17560c-0240-48c0-8889-b667b60383d6/volumes" Jul 02 19:13:40.165490 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:40.165464 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-rf6t5" Jul 02 19:13:40.232175 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:13:40.232147 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzj2c" Jul 02 19:14:10.234697 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:10.234669 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qhzkx" Jul 02 19:14:11.242356 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:11.242323 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzj2c" Jul 02 19:14:24.370760 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.370727 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9fzcn"] Jul 02 19:14:24.371166 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.371029 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa17560c-0240-48c0-8889-b667b60383d6" containerName="discovery" Jul 02 19:14:24.371166 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.371039 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa17560c-0240-48c0-8889-b667b60383d6" containerName="discovery" Jul 02 19:14:24.371166 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.371099 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa17560c-0240-48c0-8889-b667b60383d6" containerName="discovery" Jul 02 19:14:24.383310 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.383287 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9fzcn" Jul 02 19:14:24.384597 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.384569 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9fzcn"] Jul 02 19:14:24.385965 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.385946 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 19:14:24.385965 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.385960 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 19:14:24.386926 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.386905 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-mvtdp\"" Jul 02 19:14:24.467975 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.467947 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvjmm\" (UniqueName: \"kubernetes.io/projected/c940ee0e-775d-43a4-bf8f-364c2d070380-kube-api-access-rvjmm\") pod \"kuadrant-operator-catalog-9fzcn\" (UID: \"c940ee0e-775d-43a4-bf8f-364c2d070380\") " pod="kuadrant-system/kuadrant-operator-catalog-9fzcn" Jul 02 19:14:24.569195 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.569162 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvjmm\" (UniqueName: \"kubernetes.io/projected/c940ee0e-775d-43a4-bf8f-364c2d070380-kube-api-access-rvjmm\") pod \"kuadrant-operator-catalog-9fzcn\" (UID: \"c940ee0e-775d-43a4-bf8f-364c2d070380\") " pod="kuadrant-system/kuadrant-operator-catalog-9fzcn" Jul 02 19:14:24.578148 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.578120 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvjmm\" (UniqueName: \"kubernetes.io/projected/c940ee0e-775d-43a4-bf8f-364c2d070380-kube-api-access-rvjmm\") pod \"kuadrant-operator-catalog-9fzcn\" (UID: \"c940ee0e-775d-43a4-bf8f-364c2d070380\") " pod="kuadrant-system/kuadrant-operator-catalog-9fzcn" Jul 02 19:14:24.685571 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.685492 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9fzcn"] Jul 02 19:14:24.685731 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.685719 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9fzcn" Jul 02 19:14:24.816247 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.816226 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9fzcn"] Jul 02 19:14:24.818479 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:14:24.818455 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc940ee0e_775d_43a4_bf8f_364c2d070380.slice/crio-09a5883a703759a782a5c0365a497cd735a019d28ee077910036b8804eb20d2d WatchSource:0}: Error finding container 09a5883a703759a782a5c0365a497cd735a019d28ee077910036b8804eb20d2d: Status 404 returned error can't find the container with id 09a5883a703759a782a5c0365a497cd735a019d28ee077910036b8804eb20d2d Jul 02 19:14:24.896165 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.896140 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-g45f6"] Jul 02 19:14:24.900778 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.900762 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-g45f6" Jul 02 19:14:24.906465 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.906440 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-g45f6"] Jul 02 19:14:24.973705 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:24.973634 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7wkc\" (UniqueName: \"kubernetes.io/projected/7746eda0-2e7a-4e63-b8c8-80876606bbc4-kube-api-access-z7wkc\") pod \"kuadrant-operator-catalog-g45f6\" (UID: \"7746eda0-2e7a-4e63-b8c8-80876606bbc4\") " pod="kuadrant-system/kuadrant-operator-catalog-g45f6" Jul 02 19:14:25.074489 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:25.074459 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7wkc\" (UniqueName: \"kubernetes.io/projected/7746eda0-2e7a-4e63-b8c8-80876606bbc4-kube-api-access-z7wkc\") pod \"kuadrant-operator-catalog-g45f6\" (UID: \"7746eda0-2e7a-4e63-b8c8-80876606bbc4\") " pod="kuadrant-system/kuadrant-operator-catalog-g45f6" Jul 02 19:14:25.083418 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:25.083372 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7wkc\" (UniqueName: \"kubernetes.io/projected/7746eda0-2e7a-4e63-b8c8-80876606bbc4-kube-api-access-z7wkc\") pod \"kuadrant-operator-catalog-g45f6\" (UID: \"7746eda0-2e7a-4e63-b8c8-80876606bbc4\") " pod="kuadrant-system/kuadrant-operator-catalog-g45f6" Jul 02 19:14:25.211794 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:25.211765 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-g45f6" Jul 02 19:14:25.351983 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:25.351961 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-g45f6"] Jul 02 19:14:25.353709 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:14:25.353682 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7746eda0_2e7a_4e63_b8c8_80876606bbc4.slice/crio-4f66affc46b0e4b18ddd5b257162e0f02ce89c69ad4c3d8f15e4fc383f9a4118 WatchSource:0}: Error finding container 4f66affc46b0e4b18ddd5b257162e0f02ce89c69ad4c3d8f15e4fc383f9a4118: Status 404 returned error can't find the container with id 4f66affc46b0e4b18ddd5b257162e0f02ce89c69ad4c3d8f15e4fc383f9a4118 Jul 02 19:14:25.437440 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:25.437404 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-g45f6" event={"ID":"7746eda0-2e7a-4e63-b8c8-80876606bbc4","Type":"ContainerStarted","Data":"4f66affc46b0e4b18ddd5b257162e0f02ce89c69ad4c3d8f15e4fc383f9a4118"} Jul 02 19:14:25.438576 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:25.438548 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9fzcn" event={"ID":"c940ee0e-775d-43a4-bf8f-364c2d070380","Type":"ContainerStarted","Data":"09a5883a703759a782a5c0365a497cd735a019d28ee077910036b8804eb20d2d"} Jul 02 19:14:27.449926 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:27.449887 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-g45f6" event={"ID":"7746eda0-2e7a-4e63-b8c8-80876606bbc4","Type":"ContainerStarted","Data":"90887ed6fda83581ef28f634a50c1d2f553faf7fde5f418f0032d821c986a9ca"} Jul 02 19:14:27.451332 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:27.451309 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9fzcn" event={"ID":"c940ee0e-775d-43a4-bf8f-364c2d070380","Type":"ContainerStarted","Data":"119f9a160bd51a306e6909011a4d2e18fa2dc10a4734d9c7c1dd6cbff7c5f13b"} Jul 02 19:14:27.451440 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:27.451379 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-9fzcn" podUID="c940ee0e-775d-43a4-bf8f-364c2d070380" containerName="registry-server" containerID="cri-o://119f9a160bd51a306e6909011a4d2e18fa2dc10a4734d9c7c1dd6cbff7c5f13b" gracePeriod=2 Jul 02 19:14:27.466748 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:27.466711 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-g45f6" podStartSLOduration=1.938915559 podStartE2EDuration="3.466698097s" totalImagesPullingTime="1.527782538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:14:27.464702534 +0000 UTC m=+410.267517984" watchObservedRunningTime="2026-07-02 19:14:27.466698097 +0000 UTC m=+410.269513547" Jul 02 19:14:27.479871 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:27.479835 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-9fzcn" podStartSLOduration=1.4199285719999999 podStartE2EDuration="3.479823207s" totalImagesPullingTime="2.059894635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:14:27.478523563 +0000 UTC m=+410.281339017" watchObservedRunningTime="2026-07-02 19:14:27.479823207 +0000 UTC m=+410.282638657" Jul 02 19:14:27.693994 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:27.693970 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9fzcn" Jul 02 19:14:27.797856 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:27.797798 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvjmm\" (UniqueName: \"kubernetes.io/projected/c940ee0e-775d-43a4-bf8f-364c2d070380-kube-api-access-rvjmm\") pod \"c940ee0e-775d-43a4-bf8f-364c2d070380\" (UID: \"c940ee0e-775d-43a4-bf8f-364c2d070380\") " Jul 02 19:14:27.799836 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:27.799807 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c940ee0e-775d-43a4-bf8f-364c2d070380-kube-api-access-rvjmm" (OuterVolumeSpecName: "kube-api-access-rvjmm") pod "c940ee0e-775d-43a4-bf8f-364c2d070380" (UID: "c940ee0e-775d-43a4-bf8f-364c2d070380"). InnerVolumeSpecName "kube-api-access-rvjmm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:14:27.898806 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:27.898780 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvjmm\" (UniqueName: \"kubernetes.io/projected/c940ee0e-775d-43a4-bf8f-364c2d070380-kube-api-access-rvjmm\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:14:28.456895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:28.456865 2578 generic.go:356] "Generic (PLEG): container finished" podID="c940ee0e-775d-43a4-bf8f-364c2d070380" containerID="119f9a160bd51a306e6909011a4d2e18fa2dc10a4734d9c7c1dd6cbff7c5f13b" exitCode=0 Jul 02 19:14:28.457299 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:28.456909 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9fzcn" Jul 02 19:14:28.457299 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:28.456938 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9fzcn" event={"ID":"c940ee0e-775d-43a4-bf8f-364c2d070380","Type":"ContainerDied","Data":"119f9a160bd51a306e6909011a4d2e18fa2dc10a4734d9c7c1dd6cbff7c5f13b"} Jul 02 19:14:28.457299 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:28.456973 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9fzcn" event={"ID":"c940ee0e-775d-43a4-bf8f-364c2d070380","Type":"ContainerDied","Data":"09a5883a703759a782a5c0365a497cd735a019d28ee077910036b8804eb20d2d"} Jul 02 19:14:28.457299 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:28.456994 2578 scope.go:117] "RemoveContainer" containerID="119f9a160bd51a306e6909011a4d2e18fa2dc10a4734d9c7c1dd6cbff7c5f13b" Jul 02 19:14:28.466855 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:28.466838 2578 scope.go:117] "RemoveContainer" containerID="119f9a160bd51a306e6909011a4d2e18fa2dc10a4734d9c7c1dd6cbff7c5f13b" Jul 02 19:14:28.467091 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:14:28.467075 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"119f9a160bd51a306e6909011a4d2e18fa2dc10a4734d9c7c1dd6cbff7c5f13b\": container with ID starting with 119f9a160bd51a306e6909011a4d2e18fa2dc10a4734d9c7c1dd6cbff7c5f13b not found: ID does not exist" containerID="119f9a160bd51a306e6909011a4d2e18fa2dc10a4734d9c7c1dd6cbff7c5f13b" Jul 02 19:14:28.467139 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:28.467098 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"119f9a160bd51a306e6909011a4d2e18fa2dc10a4734d9c7c1dd6cbff7c5f13b"} err="failed to get container status \"119f9a160bd51a306e6909011a4d2e18fa2dc10a4734d9c7c1dd6cbff7c5f13b\": rpc error: code = NotFound desc = could not find container \"119f9a160bd51a306e6909011a4d2e18fa2dc10a4734d9c7c1dd6cbff7c5f13b\": container with ID starting with 119f9a160bd51a306e6909011a4d2e18fa2dc10a4734d9c7c1dd6cbff7c5f13b not found: ID does not exist" Jul 02 19:14:28.478587 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:28.478566 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9fzcn"] Jul 02 19:14:28.481584 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:28.481566 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9fzcn"] Jul 02 19:14:29.726137 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:29.726104 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c940ee0e-775d-43a4-bf8f-364c2d070380" path="/var/lib/kubelet/pods/c940ee0e-775d-43a4-bf8f-364c2d070380/volumes" Jul 02 19:14:35.212958 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:35.212913 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-g45f6" Jul 02 19:14:35.213369 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:35.212979 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-g45f6" Jul 02 19:14:35.235310 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:35.235287 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-g45f6" Jul 02 19:14:35.512014 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:35.511949 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-g45f6" Jul 02 19:14:54.172558 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.172517 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4"] Jul 02 19:14:54.173141 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.172990 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c940ee0e-775d-43a4-bf8f-364c2d070380" containerName="registry-server" Jul 02 19:14:54.173141 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.173010 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c940ee0e-775d-43a4-bf8f-364c2d070380" containerName="registry-server" Jul 02 19:14:54.173141 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.173111 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c940ee0e-775d-43a4-bf8f-364c2d070380" containerName="registry-server" Jul 02 19:14:54.177539 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.177516 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" Jul 02 19:14:54.179839 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.179820 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pfnst\"" Jul 02 19:14:54.185517 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.185495 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4"] Jul 02 19:14:54.274274 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.274247 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k"] Jul 02 19:14:54.277587 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.277571 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" Jul 02 19:14:54.285741 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.285712 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k"] Jul 02 19:14:54.295764 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.295744 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8912423-b85b-47fc-8969-9ca35fa01f7b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4\" (UID: \"b8912423-b85b-47fc-8969-9ca35fa01f7b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" Jul 02 19:14:54.295869 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.295773 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhrcx\" (UniqueName: \"kubernetes.io/projected/b8912423-b85b-47fc-8969-9ca35fa01f7b-kube-api-access-qhrcx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4\" (UID: \"b8912423-b85b-47fc-8969-9ca35fa01f7b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" Jul 02 19:14:54.295869 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.295797 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8912423-b85b-47fc-8969-9ca35fa01f7b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4\" (UID: \"b8912423-b85b-47fc-8969-9ca35fa01f7b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" Jul 02 19:14:54.396853 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.396829 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k\" (UID: \"a6a1bfe1-7c8e-400d-ad31-59fab76a5611\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" Jul 02 19:14:54.396972 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.396877 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v2qw\" (UniqueName: \"kubernetes.io/projected/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-kube-api-access-6v2qw\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k\" (UID: \"a6a1bfe1-7c8e-400d-ad31-59fab76a5611\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" Jul 02 19:14:54.396972 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.396962 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8912423-b85b-47fc-8969-9ca35fa01f7b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4\" (UID: \"b8912423-b85b-47fc-8969-9ca35fa01f7b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" Jul 02 19:14:54.397076 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.396988 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhrcx\" (UniqueName: \"kubernetes.io/projected/b8912423-b85b-47fc-8969-9ca35fa01f7b-kube-api-access-qhrcx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4\" (UID: \"b8912423-b85b-47fc-8969-9ca35fa01f7b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" Jul 02 19:14:54.397076 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.397014 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8912423-b85b-47fc-8969-9ca35fa01f7b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4\" (UID: \"b8912423-b85b-47fc-8969-9ca35fa01f7b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" Jul 02 19:14:54.397076 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.397033 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k\" (UID: \"a6a1bfe1-7c8e-400d-ad31-59fab76a5611\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" Jul 02 19:14:54.397337 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.397319 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8912423-b85b-47fc-8969-9ca35fa01f7b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4\" (UID: \"b8912423-b85b-47fc-8969-9ca35fa01f7b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" Jul 02 19:14:54.397412 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.397343 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8912423-b85b-47fc-8969-9ca35fa01f7b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4\" (UID: \"b8912423-b85b-47fc-8969-9ca35fa01f7b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" Jul 02 19:14:54.406265 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.406245 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhrcx\" (UniqueName: \"kubernetes.io/projected/b8912423-b85b-47fc-8969-9ca35fa01f7b-kube-api-access-qhrcx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4\" (UID: \"b8912423-b85b-47fc-8969-9ca35fa01f7b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" Jul 02 19:14:54.487206 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.487150 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" Jul 02 19:14:54.498218 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.498198 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k\" (UID: \"a6a1bfe1-7c8e-400d-ad31-59fab76a5611\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" Jul 02 19:14:54.498287 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.498235 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v2qw\" (UniqueName: \"kubernetes.io/projected/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-kube-api-access-6v2qw\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k\" (UID: \"a6a1bfe1-7c8e-400d-ad31-59fab76a5611\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" Jul 02 19:14:54.498287 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.498270 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k\" (UID: \"a6a1bfe1-7c8e-400d-ad31-59fab76a5611\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" Jul 02 19:14:54.498623 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.498605 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k\" (UID: \"a6a1bfe1-7c8e-400d-ad31-59fab76a5611\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" Jul 02 19:14:54.498689 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.498648 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k\" (UID: \"a6a1bfe1-7c8e-400d-ad31-59fab76a5611\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" Jul 02 19:14:54.507038 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.507014 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v2qw\" (UniqueName: \"kubernetes.io/projected/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-kube-api-access-6v2qw\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k\" (UID: \"a6a1bfe1-7c8e-400d-ad31-59fab76a5611\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" Jul 02 19:14:54.573563 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.573527 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j"] Jul 02 19:14:54.578071 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.578047 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" Jul 02 19:14:54.588075 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.588041 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" Jul 02 19:14:54.590365 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.590316 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j"] Jul 02 19:14:54.627131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.627104 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4"] Jul 02 19:14:54.629223 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:14:54.629201 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8912423_b85b_47fc_8969_9ca35fa01f7b.slice/crio-4e21948aaa9e854f874c4b995b29e0fd67a7767737f4624891972812aea1a568 WatchSource:0}: Error finding container 4e21948aaa9e854f874c4b995b29e0fd67a7767737f4624891972812aea1a568: Status 404 returned error can't find the container with id 4e21948aaa9e854f874c4b995b29e0fd67a7767737f4624891972812aea1a568 Jul 02 19:14:54.700032 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.699897 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7da25c73-a292-4803-8fa5-58d9773329dd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j\" (UID: \"7da25c73-a292-4803-8fa5-58d9773329dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" Jul 02 19:14:54.700032 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.699952 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrdn9\" (UniqueName: \"kubernetes.io/projected/7da25c73-a292-4803-8fa5-58d9773329dd-kube-api-access-nrdn9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j\" (UID: \"7da25c73-a292-4803-8fa5-58d9773329dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" Jul 02 19:14:54.700032 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.699990 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7da25c73-a292-4803-8fa5-58d9773329dd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j\" (UID: \"7da25c73-a292-4803-8fa5-58d9773329dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" Jul 02 19:14:54.722373 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.722351 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k"] Jul 02 19:14:54.722870 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:14:54.722838 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6a1bfe1_7c8e_400d_ad31_59fab76a5611.slice/crio-795a9c6c7d2b4dfc043501035e0f455a4bd40634c3e58de8896681d2cdf26620 WatchSource:0}: Error finding container 795a9c6c7d2b4dfc043501035e0f455a4bd40634c3e58de8896681d2cdf26620: Status 404 returned error can't find the container with id 795a9c6c7d2b4dfc043501035e0f455a4bd40634c3e58de8896681d2cdf26620 Jul 02 19:14:54.801476 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.801447 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7da25c73-a292-4803-8fa5-58d9773329dd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j\" (UID: \"7da25c73-a292-4803-8fa5-58d9773329dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" Jul 02 19:14:54.801616 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.801492 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrdn9\" (UniqueName: \"kubernetes.io/projected/7da25c73-a292-4803-8fa5-58d9773329dd-kube-api-access-nrdn9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j\" (UID: \"7da25c73-a292-4803-8fa5-58d9773329dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" Jul 02 19:14:54.801616 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.801545 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7da25c73-a292-4803-8fa5-58d9773329dd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j\" (UID: \"7da25c73-a292-4803-8fa5-58d9773329dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" Jul 02 19:14:54.801890 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.801870 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7da25c73-a292-4803-8fa5-58d9773329dd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j\" (UID: \"7da25c73-a292-4803-8fa5-58d9773329dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" Jul 02 19:14:54.801962 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.801899 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7da25c73-a292-4803-8fa5-58d9773329dd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j\" (UID: \"7da25c73-a292-4803-8fa5-58d9773329dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" Jul 02 19:14:54.810553 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.810529 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrdn9\" (UniqueName: \"kubernetes.io/projected/7da25c73-a292-4803-8fa5-58d9773329dd-kube-api-access-nrdn9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j\" (UID: \"7da25c73-a292-4803-8fa5-58d9773329dd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" Jul 02 19:14:54.890212 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.890187 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" Jul 02 19:14:54.975775 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.975746 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5"] Jul 02 19:14:54.989156 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.989128 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5"] Jul 02 19:14:54.989328 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:54.989232 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" Jul 02 19:14:55.018244 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.018187 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j"] Jul 02 19:14:55.021041 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:14:55.021023 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7da25c73_a292_4803_8fa5_58d9773329dd.slice/crio-f7c8c712b87bd344a9856c29612d754d45cc2afd6b588ed8f8648e9bcc39acc3 WatchSource:0}: Error finding container f7c8c712b87bd344a9856c29612d754d45cc2afd6b588ed8f8648e9bcc39acc3: Status 404 returned error can't find the container with id f7c8c712b87bd344a9856c29612d754d45cc2afd6b588ed8f8648e9bcc39acc3 Jul 02 19:14:55.103637 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.103609 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0131c51-c13a-4538-aa05-16171908f1bc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5\" (UID: \"e0131c51-c13a-4538-aa05-16171908f1bc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" Jul 02 19:14:55.103790 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.103671 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54xfx\" (UniqueName: \"kubernetes.io/projected/e0131c51-c13a-4538-aa05-16171908f1bc-kube-api-access-54xfx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5\" (UID: \"e0131c51-c13a-4538-aa05-16171908f1bc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" Jul 02 19:14:55.103790 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.103729 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0131c51-c13a-4538-aa05-16171908f1bc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5\" (UID: \"e0131c51-c13a-4538-aa05-16171908f1bc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" Jul 02 19:14:55.204427 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.204380 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0131c51-c13a-4538-aa05-16171908f1bc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5\" (UID: \"e0131c51-c13a-4538-aa05-16171908f1bc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" Jul 02 19:14:55.204735 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.204444 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54xfx\" (UniqueName: \"kubernetes.io/projected/e0131c51-c13a-4538-aa05-16171908f1bc-kube-api-access-54xfx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5\" (UID: \"e0131c51-c13a-4538-aa05-16171908f1bc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" Jul 02 19:14:55.204735 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.204474 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0131c51-c13a-4538-aa05-16171908f1bc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5\" (UID: \"e0131c51-c13a-4538-aa05-16171908f1bc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" Jul 02 19:14:55.204735 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.204722 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0131c51-c13a-4538-aa05-16171908f1bc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5\" (UID: \"e0131c51-c13a-4538-aa05-16171908f1bc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" Jul 02 19:14:55.204735 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.204735 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0131c51-c13a-4538-aa05-16171908f1bc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5\" (UID: \"e0131c51-c13a-4538-aa05-16171908f1bc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" Jul 02 19:14:55.211917 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.211896 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54xfx\" (UniqueName: \"kubernetes.io/projected/e0131c51-c13a-4538-aa05-16171908f1bc-kube-api-access-54xfx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5\" (UID: \"e0131c51-c13a-4538-aa05-16171908f1bc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" Jul 02 19:14:55.300256 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.300178 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" Jul 02 19:14:55.426016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.425994 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5"] Jul 02 19:14:55.428544 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:14:55.428517 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0131c51_c13a_4538_aa05_16171908f1bc.slice/crio-590e5dcaad70787555dfd62dc48817bb6857a40ac96544aa9d9f4d29a54c09b6 WatchSource:0}: Error finding container 590e5dcaad70787555dfd62dc48817bb6857a40ac96544aa9d9f4d29a54c09b6: Status 404 returned error can't find the container with id 590e5dcaad70787555dfd62dc48817bb6857a40ac96544aa9d9f4d29a54c09b6 Jul 02 19:14:55.580235 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.580210 2578 generic.go:356] "Generic (PLEG): container finished" podID="a6a1bfe1-7c8e-400d-ad31-59fab76a5611" containerID="0ae4c5c85d32d96c030f142342feb1779706dd7ab0b021d66493655f698f211a" exitCode=0 Jul 02 19:14:55.580347 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.580268 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" event={"ID":"a6a1bfe1-7c8e-400d-ad31-59fab76a5611","Type":"ContainerDied","Data":"0ae4c5c85d32d96c030f142342feb1779706dd7ab0b021d66493655f698f211a"} Jul 02 19:14:55.580347 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.580321 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" event={"ID":"a6a1bfe1-7c8e-400d-ad31-59fab76a5611","Type":"ContainerStarted","Data":"795a9c6c7d2b4dfc043501035e0f455a4bd40634c3e58de8896681d2cdf26620"} Jul 02 19:14:55.582018 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.581977 2578 generic.go:356] "Generic (PLEG): container finished" podID="b8912423-b85b-47fc-8969-9ca35fa01f7b" containerID="cb60891a591006e46f1c31b498cc2a3ee600ec3268adcf747787d654fdbef86b" exitCode=0 Jul 02 19:14:55.582122 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.582028 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" event={"ID":"b8912423-b85b-47fc-8969-9ca35fa01f7b","Type":"ContainerDied","Data":"cb60891a591006e46f1c31b498cc2a3ee600ec3268adcf747787d654fdbef86b"} Jul 02 19:14:55.582122 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.582068 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" event={"ID":"b8912423-b85b-47fc-8969-9ca35fa01f7b","Type":"ContainerStarted","Data":"4e21948aaa9e854f874c4b995b29e0fd67a7767737f4624891972812aea1a568"} Jul 02 19:14:55.583777 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.583754 2578 generic.go:356] "Generic (PLEG): container finished" podID="7da25c73-a292-4803-8fa5-58d9773329dd" containerID="dab184b4465ffd088eb9a093bc57975013b3c79318e1876fa54fe599f0ca41f2" exitCode=0 Jul 02 19:14:55.583869 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.583839 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" event={"ID":"7da25c73-a292-4803-8fa5-58d9773329dd","Type":"ContainerDied","Data":"dab184b4465ffd088eb9a093bc57975013b3c79318e1876fa54fe599f0ca41f2"} Jul 02 19:14:55.583939 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.583882 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" event={"ID":"7da25c73-a292-4803-8fa5-58d9773329dd","Type":"ContainerStarted","Data":"f7c8c712b87bd344a9856c29612d754d45cc2afd6b588ed8f8648e9bcc39acc3"} Jul 02 19:14:55.585384 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.585366 2578 generic.go:356] "Generic (PLEG): container finished" podID="e0131c51-c13a-4538-aa05-16171908f1bc" containerID="1e80301f71937b90980a140ddd90c43813fdd4a1b8dd9a71cea6d12259924fd5" exitCode=0 Jul 02 19:14:55.585496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.585429 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" event={"ID":"e0131c51-c13a-4538-aa05-16171908f1bc","Type":"ContainerDied","Data":"1e80301f71937b90980a140ddd90c43813fdd4a1b8dd9a71cea6d12259924fd5"} Jul 02 19:14:55.585496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:55.585470 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" event={"ID":"e0131c51-c13a-4538-aa05-16171908f1bc","Type":"ContainerStarted","Data":"590e5dcaad70787555dfd62dc48817bb6857a40ac96544aa9d9f4d29a54c09b6"} Jul 02 19:14:56.594820 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:56.594785 2578 generic.go:356] "Generic (PLEG): container finished" podID="a6a1bfe1-7c8e-400d-ad31-59fab76a5611" containerID="e9c0ef8010f9accc7591087372b8dd7cc1ad06175c0087e88da260bc6105d441" exitCode=0 Jul 02 19:14:56.595276 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:56.594877 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" event={"ID":"a6a1bfe1-7c8e-400d-ad31-59fab76a5611","Type":"ContainerDied","Data":"e9c0ef8010f9accc7591087372b8dd7cc1ad06175c0087e88da260bc6105d441"} Jul 02 19:14:56.599884 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:56.599860 2578 generic.go:356] "Generic (PLEG): container finished" podID="b8912423-b85b-47fc-8969-9ca35fa01f7b" containerID="29fa57830f28bc3d76aff49731ef97cb484adf75c5ce0ea559ec4536483b4a61" exitCode=0 Jul 02 19:14:56.599983 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:56.599918 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" event={"ID":"b8912423-b85b-47fc-8969-9ca35fa01f7b","Type":"ContainerDied","Data":"29fa57830f28bc3d76aff49731ef97cb484adf75c5ce0ea559ec4536483b4a61"} Jul 02 19:14:57.607159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:57.607125 2578 generic.go:356] "Generic (PLEG): container finished" podID="e0131c51-c13a-4538-aa05-16171908f1bc" containerID="9817eb4778b402ac14e1b788bafc5ddb6b0ed4557e7e8c1cd87cdfbbc73b53e4" exitCode=0 Jul 02 19:14:57.607622 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:57.607122 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" event={"ID":"e0131c51-c13a-4538-aa05-16171908f1bc","Type":"ContainerDied","Data":"9817eb4778b402ac14e1b788bafc5ddb6b0ed4557e7e8c1cd87cdfbbc73b53e4"} Jul 02 19:14:57.609539 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:57.609519 2578 generic.go:356] "Generic (PLEG): container finished" podID="a6a1bfe1-7c8e-400d-ad31-59fab76a5611" containerID="8aff0a04bf1c1e6443c3b75ae908181a4ec7a6fc4dc79b9f56b76d9d8af4250e" exitCode=0 Jul 02 19:14:57.609636 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:57.609581 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" event={"ID":"a6a1bfe1-7c8e-400d-ad31-59fab76a5611","Type":"ContainerDied","Data":"8aff0a04bf1c1e6443c3b75ae908181a4ec7a6fc4dc79b9f56b76d9d8af4250e"} Jul 02 19:14:57.611566 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:57.611548 2578 generic.go:356] "Generic (PLEG): container finished" podID="b8912423-b85b-47fc-8969-9ca35fa01f7b" containerID="30251b8fc342b569d9b0b3c286296f1b08a3f8f534c72cfe665d5824c847e8df" exitCode=0 Jul 02 19:14:57.611651 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:57.611628 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" event={"ID":"b8912423-b85b-47fc-8969-9ca35fa01f7b","Type":"ContainerDied","Data":"30251b8fc342b569d9b0b3c286296f1b08a3f8f534c72cfe665d5824c847e8df"} Jul 02 19:14:57.613376 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:57.613357 2578 generic.go:356] "Generic (PLEG): container finished" podID="7da25c73-a292-4803-8fa5-58d9773329dd" containerID="365b7ef21a1cba72bed0e222fc184f94ee2f3aff6d944dab4fc3d699e8a6dc2e" exitCode=0 Jul 02 19:14:57.613495 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:57.613428 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" event={"ID":"7da25c73-a292-4803-8fa5-58d9773329dd","Type":"ContainerDied","Data":"365b7ef21a1cba72bed0e222fc184f94ee2f3aff6d944dab4fc3d699e8a6dc2e"} Jul 02 19:14:58.620414 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.620368 2578 generic.go:356] "Generic (PLEG): container finished" podID="7da25c73-a292-4803-8fa5-58d9773329dd" containerID="341dd10001457d7d6d99cd11edc8cfc74ed4dd36c77e31b118476c2440cb7f05" exitCode=0 Jul 02 19:14:58.620839 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.620461 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" event={"ID":"7da25c73-a292-4803-8fa5-58d9773329dd","Type":"ContainerDied","Data":"341dd10001457d7d6d99cd11edc8cfc74ed4dd36c77e31b118476c2440cb7f05"} Jul 02 19:14:58.622558 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.622535 2578 generic.go:356] "Generic (PLEG): container finished" podID="e0131c51-c13a-4538-aa05-16171908f1bc" containerID="be8ae5b1453bdce98969fbceb28d400f671b7353f1fb9e4f0cba7e68eeb9c238" exitCode=0 Jul 02 19:14:58.622693 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.622599 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" event={"ID":"e0131c51-c13a-4538-aa05-16171908f1bc","Type":"ContainerDied","Data":"be8ae5b1453bdce98969fbceb28d400f671b7353f1fb9e4f0cba7e68eeb9c238"} Jul 02 19:14:58.762563 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.762541 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" Jul 02 19:14:58.791699 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.791679 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" Jul 02 19:14:58.936079 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.935987 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8912423-b85b-47fc-8969-9ca35fa01f7b-bundle\") pod \"b8912423-b85b-47fc-8969-9ca35fa01f7b\" (UID: \"b8912423-b85b-47fc-8969-9ca35fa01f7b\") " Jul 02 19:14:58.936079 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.936047 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8912423-b85b-47fc-8969-9ca35fa01f7b-util\") pod \"b8912423-b85b-47fc-8969-9ca35fa01f7b\" (UID: \"b8912423-b85b-47fc-8969-9ca35fa01f7b\") " Jul 02 19:14:58.936079 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.936067 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhrcx\" (UniqueName: \"kubernetes.io/projected/b8912423-b85b-47fc-8969-9ca35fa01f7b-kube-api-access-qhrcx\") pod \"b8912423-b85b-47fc-8969-9ca35fa01f7b\" (UID: \"b8912423-b85b-47fc-8969-9ca35fa01f7b\") " Jul 02 19:14:58.936372 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.936099 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v2qw\" (UniqueName: \"kubernetes.io/projected/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-kube-api-access-6v2qw\") pod \"a6a1bfe1-7c8e-400d-ad31-59fab76a5611\" (UID: \"a6a1bfe1-7c8e-400d-ad31-59fab76a5611\") " Jul 02 19:14:58.936372 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.936137 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-bundle\") pod \"a6a1bfe1-7c8e-400d-ad31-59fab76a5611\" (UID: \"a6a1bfe1-7c8e-400d-ad31-59fab76a5611\") " Jul 02 19:14:58.936372 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.936171 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-util\") pod \"a6a1bfe1-7c8e-400d-ad31-59fab76a5611\" (UID: \"a6a1bfe1-7c8e-400d-ad31-59fab76a5611\") " Jul 02 19:14:58.936697 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.936649 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8912423-b85b-47fc-8969-9ca35fa01f7b-bundle" (OuterVolumeSpecName: "bundle") pod "b8912423-b85b-47fc-8969-9ca35fa01f7b" (UID: "b8912423-b85b-47fc-8969-9ca35fa01f7b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:14:58.936942 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.936913 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-bundle" (OuterVolumeSpecName: "bundle") pod "a6a1bfe1-7c8e-400d-ad31-59fab76a5611" (UID: "a6a1bfe1-7c8e-400d-ad31-59fab76a5611"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:14:58.938744 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.938708 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8912423-b85b-47fc-8969-9ca35fa01f7b-kube-api-access-qhrcx" (OuterVolumeSpecName: "kube-api-access-qhrcx") pod "b8912423-b85b-47fc-8969-9ca35fa01f7b" (UID: "b8912423-b85b-47fc-8969-9ca35fa01f7b"). InnerVolumeSpecName "kube-api-access-qhrcx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:14:58.938828 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.938797 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-kube-api-access-6v2qw" (OuterVolumeSpecName: "kube-api-access-6v2qw") pod "a6a1bfe1-7c8e-400d-ad31-59fab76a5611" (UID: "a6a1bfe1-7c8e-400d-ad31-59fab76a5611"). InnerVolumeSpecName "kube-api-access-6v2qw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:14:58.941915 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.941876 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-util" (OuterVolumeSpecName: "util") pod "a6a1bfe1-7c8e-400d-ad31-59fab76a5611" (UID: "a6a1bfe1-7c8e-400d-ad31-59fab76a5611"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:14:58.942381 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:58.942364 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8912423-b85b-47fc-8969-9ca35fa01f7b-util" (OuterVolumeSpecName: "util") pod "b8912423-b85b-47fc-8969-9ca35fa01f7b" (UID: "b8912423-b85b-47fc-8969-9ca35fa01f7b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:14:59.036703 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.036682 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-util\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:14:59.036703 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.036702 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8912423-b85b-47fc-8969-9ca35fa01f7b-bundle\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:14:59.036851 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.036711 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8912423-b85b-47fc-8969-9ca35fa01f7b-util\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:14:59.036851 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.036719 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhrcx\" (UniqueName: \"kubernetes.io/projected/b8912423-b85b-47fc-8969-9ca35fa01f7b-kube-api-access-qhrcx\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:14:59.036851 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.036728 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v2qw\" (UniqueName: \"kubernetes.io/projected/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-kube-api-access-6v2qw\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:14:59.036851 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.036736 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6a1bfe1-7c8e-400d-ad31-59fab76a5611-bundle\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:14:59.629272 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.629240 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" event={"ID":"a6a1bfe1-7c8e-400d-ad31-59fab76a5611","Type":"ContainerDied","Data":"795a9c6c7d2b4dfc043501035e0f455a4bd40634c3e58de8896681d2cdf26620"} Jul 02 19:14:59.629272 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.629260 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k" Jul 02 19:14:59.629763 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.629288 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="795a9c6c7d2b4dfc043501035e0f455a4bd40634c3e58de8896681d2cdf26620" Jul 02 19:14:59.631098 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.631080 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" Jul 02 19:14:59.631199 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.631115 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4" event={"ID":"b8912423-b85b-47fc-8969-9ca35fa01f7b","Type":"ContainerDied","Data":"4e21948aaa9e854f874c4b995b29e0fd67a7767737f4624891972812aea1a568"} Jul 02 19:14:59.631199 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.631146 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e21948aaa9e854f874c4b995b29e0fd67a7767737f4624891972812aea1a568" Jul 02 19:14:59.776246 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.776216 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" Jul 02 19:14:59.778198 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.778181 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" Jul 02 19:14:59.942526 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.942446 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54xfx\" (UniqueName: \"kubernetes.io/projected/e0131c51-c13a-4538-aa05-16171908f1bc-kube-api-access-54xfx\") pod \"e0131c51-c13a-4538-aa05-16171908f1bc\" (UID: \"e0131c51-c13a-4538-aa05-16171908f1bc\") " Jul 02 19:14:59.942526 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.942492 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7da25c73-a292-4803-8fa5-58d9773329dd-util\") pod \"7da25c73-a292-4803-8fa5-58d9773329dd\" (UID: \"7da25c73-a292-4803-8fa5-58d9773329dd\") " Jul 02 19:14:59.942526 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.942518 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrdn9\" (UniqueName: \"kubernetes.io/projected/7da25c73-a292-4803-8fa5-58d9773329dd-kube-api-access-nrdn9\") pod \"7da25c73-a292-4803-8fa5-58d9773329dd\" (UID: \"7da25c73-a292-4803-8fa5-58d9773329dd\") " Jul 02 19:14:59.942777 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.942551 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0131c51-c13a-4538-aa05-16171908f1bc-bundle\") pod \"e0131c51-c13a-4538-aa05-16171908f1bc\" (UID: \"e0131c51-c13a-4538-aa05-16171908f1bc\") " Jul 02 19:14:59.942777 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.942599 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0131c51-c13a-4538-aa05-16171908f1bc-util\") pod \"e0131c51-c13a-4538-aa05-16171908f1bc\" (UID: \"e0131c51-c13a-4538-aa05-16171908f1bc\") " Jul 02 19:14:59.942777 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.942644 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7da25c73-a292-4803-8fa5-58d9773329dd-bundle\") pod \"7da25c73-a292-4803-8fa5-58d9773329dd\" (UID: \"7da25c73-a292-4803-8fa5-58d9773329dd\") " Jul 02 19:14:59.943208 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.943160 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0131c51-c13a-4538-aa05-16171908f1bc-bundle" (OuterVolumeSpecName: "bundle") pod "e0131c51-c13a-4538-aa05-16171908f1bc" (UID: "e0131c51-c13a-4538-aa05-16171908f1bc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:14:59.943208 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.943166 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da25c73-a292-4803-8fa5-58d9773329dd-bundle" (OuterVolumeSpecName: "bundle") pod "7da25c73-a292-4803-8fa5-58d9773329dd" (UID: "7da25c73-a292-4803-8fa5-58d9773329dd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:14:59.944656 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.944632 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0131c51-c13a-4538-aa05-16171908f1bc-kube-api-access-54xfx" (OuterVolumeSpecName: "kube-api-access-54xfx") pod "e0131c51-c13a-4538-aa05-16171908f1bc" (UID: "e0131c51-c13a-4538-aa05-16171908f1bc"). InnerVolumeSpecName "kube-api-access-54xfx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:14:59.944946 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.944925 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7da25c73-a292-4803-8fa5-58d9773329dd-kube-api-access-nrdn9" (OuterVolumeSpecName: "kube-api-access-nrdn9") pod "7da25c73-a292-4803-8fa5-58d9773329dd" (UID: "7da25c73-a292-4803-8fa5-58d9773329dd"). InnerVolumeSpecName "kube-api-access-nrdn9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:14:59.948261 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.948231 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da25c73-a292-4803-8fa5-58d9773329dd-util" (OuterVolumeSpecName: "util") pod "7da25c73-a292-4803-8fa5-58d9773329dd" (UID: "7da25c73-a292-4803-8fa5-58d9773329dd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:14:59.948442 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:14:59.948423 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0131c51-c13a-4538-aa05-16171908f1bc-util" (OuterVolumeSpecName: "util") pod "e0131c51-c13a-4538-aa05-16171908f1bc" (UID: "e0131c51-c13a-4538-aa05-16171908f1bc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:15:00.043513 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:00.043488 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0131c51-c13a-4538-aa05-16171908f1bc-util\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:15:00.043513 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:00.043507 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7da25c73-a292-4803-8fa5-58d9773329dd-bundle\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:15:00.043513 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:00.043516 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54xfx\" (UniqueName: \"kubernetes.io/projected/e0131c51-c13a-4538-aa05-16171908f1bc-kube-api-access-54xfx\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:15:00.043693 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:00.043527 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7da25c73-a292-4803-8fa5-58d9773329dd-util\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:15:00.043693 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:00.043536 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrdn9\" (UniqueName: \"kubernetes.io/projected/7da25c73-a292-4803-8fa5-58d9773329dd-kube-api-access-nrdn9\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:15:00.043693 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:00.043544 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0131c51-c13a-4538-aa05-16171908f1bc-bundle\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:15:00.643212 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:00.643180 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" Jul 02 19:15:00.643645 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:00.643183 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5" event={"ID":"e0131c51-c13a-4538-aa05-16171908f1bc","Type":"ContainerDied","Data":"590e5dcaad70787555dfd62dc48817bb6857a40ac96544aa9d9f4d29a54c09b6"} Jul 02 19:15:00.643645 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:00.643376 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="590e5dcaad70787555dfd62dc48817bb6857a40ac96544aa9d9f4d29a54c09b6" Jul 02 19:15:00.645189 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:00.645164 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" event={"ID":"7da25c73-a292-4803-8fa5-58d9773329dd","Type":"ContainerDied","Data":"f7c8c712b87bd344a9856c29612d754d45cc2afd6b588ed8f8648e9bcc39acc3"} Jul 02 19:15:00.645311 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:00.645212 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7c8c712b87bd344a9856c29612d754d45cc2afd6b588ed8f8648e9bcc39acc3" Jul 02 19:15:00.645311 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:00.645215 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j" Jul 02 19:15:09.678526 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678493 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v"] Jul 02 19:15:09.678895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678793 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6a1bfe1-7c8e-400d-ad31-59fab76a5611" containerName="pull" Jul 02 19:15:09.678895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678804 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6a1bfe1-7c8e-400d-ad31-59fab76a5611" containerName="pull" Jul 02 19:15:09.678895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678815 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8912423-b85b-47fc-8969-9ca35fa01f7b" containerName="pull" Jul 02 19:15:09.678895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678821 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8912423-b85b-47fc-8969-9ca35fa01f7b" containerName="pull" Jul 02 19:15:09.678895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678827 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7da25c73-a292-4803-8fa5-58d9773329dd" containerName="pull" Jul 02 19:15:09.678895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678834 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7da25c73-a292-4803-8fa5-58d9773329dd" containerName="pull" Jul 02 19:15:09.678895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678847 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8912423-b85b-47fc-8969-9ca35fa01f7b" containerName="extract" Jul 02 19:15:09.678895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678852 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8912423-b85b-47fc-8969-9ca35fa01f7b" containerName="extract" Jul 02 19:15:09.678895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678859 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0131c51-c13a-4538-aa05-16171908f1bc" containerName="pull" Jul 02 19:15:09.678895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678864 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0131c51-c13a-4538-aa05-16171908f1bc" containerName="pull" Jul 02 19:15:09.678895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678872 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8912423-b85b-47fc-8969-9ca35fa01f7b" containerName="util" Jul 02 19:15:09.678895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678877 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8912423-b85b-47fc-8969-9ca35fa01f7b" containerName="util" Jul 02 19:15:09.678895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678883 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6a1bfe1-7c8e-400d-ad31-59fab76a5611" containerName="extract" Jul 02 19:15:09.678895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678887 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6a1bfe1-7c8e-400d-ad31-59fab76a5611" containerName="extract" Jul 02 19:15:09.678895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678893 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0131c51-c13a-4538-aa05-16171908f1bc" containerName="extract" Jul 02 19:15:09.678895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678897 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0131c51-c13a-4538-aa05-16171908f1bc" containerName="extract" Jul 02 19:15:09.678895 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678903 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7da25c73-a292-4803-8fa5-58d9773329dd" containerName="extract" Jul 02 19:15:09.679450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678908 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7da25c73-a292-4803-8fa5-58d9773329dd" containerName="extract" Jul 02 19:15:09.679450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678914 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6a1bfe1-7c8e-400d-ad31-59fab76a5611" containerName="util" Jul 02 19:15:09.679450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678918 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6a1bfe1-7c8e-400d-ad31-59fab76a5611" containerName="util" Jul 02 19:15:09.679450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678924 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7da25c73-a292-4803-8fa5-58d9773329dd" containerName="util" Jul 02 19:15:09.679450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678929 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7da25c73-a292-4803-8fa5-58d9773329dd" containerName="util" Jul 02 19:15:09.679450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678937 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0131c51-c13a-4538-aa05-16171908f1bc" containerName="util" Jul 02 19:15:09.679450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678941 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0131c51-c13a-4538-aa05-16171908f1bc" containerName="util" Jul 02 19:15:09.679450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.678990 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0131c51-c13a-4538-aa05-16171908f1bc" containerName="extract" Jul 02 19:15:09.679450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.679000 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7da25c73-a292-4803-8fa5-58d9773329dd" containerName="extract" Jul 02 19:15:09.679450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.679005 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6a1bfe1-7c8e-400d-ad31-59fab76a5611" containerName="extract" Jul 02 19:15:09.679450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.679011 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8912423-b85b-47fc-8969-9ca35fa01f7b" containerName="extract" Jul 02 19:15:09.682729 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.682708 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" Jul 02 19:15:09.685457 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.685434 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-54kx6\"" Jul 02 19:15:09.698070 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.698048 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v"] Jul 02 19:15:09.726185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.726161 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs68t\" (UniqueName: \"kubernetes.io/projected/6f77cf09-a4ca-484c-bded-4c5775d29e2b-kube-api-access-bs68t\") pod \"limitador-operator-controller-manager-859d7b78bd-q6p7v\" (UID: \"6f77cf09-a4ca-484c-bded-4c5775d29e2b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" Jul 02 19:15:09.827133 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.827093 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs68t\" (UniqueName: \"kubernetes.io/projected/6f77cf09-a4ca-484c-bded-4c5775d29e2b-kube-api-access-bs68t\") pod \"limitador-operator-controller-manager-859d7b78bd-q6p7v\" (UID: \"6f77cf09-a4ca-484c-bded-4c5775d29e2b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" Jul 02 19:15:09.835208 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.835178 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs68t\" (UniqueName: \"kubernetes.io/projected/6f77cf09-a4ca-484c-bded-4c5775d29e2b-kube-api-access-bs68t\") pod \"limitador-operator-controller-manager-859d7b78bd-q6p7v\" (UID: \"6f77cf09-a4ca-484c-bded-4c5775d29e2b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" Jul 02 19:15:09.992687 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:09.992577 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" Jul 02 19:15:10.125355 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:10.125333 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v"] Jul 02 19:15:10.127342 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:15:10.127318 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f77cf09_a4ca_484c_bded_4c5775d29e2b.slice/crio-899afc96a910134aa5159330625b5fe0502e39ee03251b241f673b10063fcd61 WatchSource:0}: Error finding container 899afc96a910134aa5159330625b5fe0502e39ee03251b241f673b10063fcd61: Status 404 returned error can't find the container with id 899afc96a910134aa5159330625b5fe0502e39ee03251b241f673b10063fcd61 Jul 02 19:15:10.692540 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:10.692498 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" event={"ID":"6f77cf09-a4ca-484c-bded-4c5775d29e2b","Type":"ContainerStarted","Data":"899afc96a910134aa5159330625b5fe0502e39ee03251b241f673b10063fcd61"} Jul 02 19:15:12.703465 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:12.703419 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" event={"ID":"6f77cf09-a4ca-484c-bded-4c5775d29e2b","Type":"ContainerStarted","Data":"34dadada5d95b9ad8023888daf89386f13dfe30b75d06cd70137a151f0a24b1b"} Jul 02 19:15:12.703854 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:12.703528 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" Jul 02 19:15:13.091542 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:13.091490 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" podStartSLOduration=1.989000168 podStartE2EDuration="4.091472886s" totalImagesPullingTime="2.102472718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:15:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:15:12.72208919 +0000 UTC m=+455.524904661" watchObservedRunningTime="2026-07-02 19:15:13.091472886 +0000 UTC m=+455.894288337" Jul 02 19:15:13.094436 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:13.093265 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-n9txz"] Jul 02 19:15:13.098085 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:13.098056 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n9txz" Jul 02 19:15:13.100359 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:13.100341 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-n5mwn\"" Jul 02 19:15:13.100471 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:13.100377 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 02 19:15:13.106071 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:13.106051 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-n9txz"] Jul 02 19:15:13.154938 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:13.154915 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmq9w\" (UniqueName: \"kubernetes.io/projected/2c42f01d-d3bb-47b2-8944-ffd9d0838c89-kube-api-access-fmq9w\") pod \"dns-operator-controller-manager-864dd78c4c-n9txz\" (UID: \"2c42f01d-d3bb-47b2-8944-ffd9d0838c89\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n9txz" Jul 02 19:15:13.255703 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:13.255677 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmq9w\" (UniqueName: \"kubernetes.io/projected/2c42f01d-d3bb-47b2-8944-ffd9d0838c89-kube-api-access-fmq9w\") pod \"dns-operator-controller-manager-864dd78c4c-n9txz\" (UID: \"2c42f01d-d3bb-47b2-8944-ffd9d0838c89\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n9txz" Jul 02 19:15:13.265616 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:13.265597 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmq9w\" (UniqueName: \"kubernetes.io/projected/2c42f01d-d3bb-47b2-8944-ffd9d0838c89-kube-api-access-fmq9w\") pod \"dns-operator-controller-manager-864dd78c4c-n9txz\" (UID: \"2c42f01d-d3bb-47b2-8944-ffd9d0838c89\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n9txz" Jul 02 19:15:13.408194 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:13.408134 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n9txz" Jul 02 19:15:13.537910 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:13.537887 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-n9txz"] Jul 02 19:15:13.539939 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:15:13.539909 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c42f01d_d3bb_47b2_8944_ffd9d0838c89.slice/crio-5bb5e49c32a8b1684700cf8ff015d174a0844008e3002e322235516f7d0c2267 WatchSource:0}: Error finding container 5bb5e49c32a8b1684700cf8ff015d174a0844008e3002e322235516f7d0c2267: Status 404 returned error can't find the container with id 5bb5e49c32a8b1684700cf8ff015d174a0844008e3002e322235516f7d0c2267 Jul 02 19:15:13.709608 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:13.709536 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n9txz" event={"ID":"2c42f01d-d3bb-47b2-8944-ffd9d0838c89","Type":"ContainerStarted","Data":"5bb5e49c32a8b1684700cf8ff015d174a0844008e3002e322235516f7d0c2267"} Jul 02 19:15:15.726531 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:15.726500 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n9txz" event={"ID":"2c42f01d-d3bb-47b2-8944-ffd9d0838c89","Type":"ContainerStarted","Data":"ad170e1080407d1c0cff7752293eaef9e7d7bdff3b225e689cb1e39dae745238"} Jul 02 19:15:15.726846 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:15.726552 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n9txz" Jul 02 19:15:15.743073 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:15.743033 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n9txz" podStartSLOduration=0.726928887 podStartE2EDuration="2.743014999s" totalImagesPullingTime="2.016086112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:15:15.741470911 +0000 UTC m=+458.544286383" watchObservedRunningTime="2026-07-02 19:15:15.743014999 +0000 UTC m=+458.545830510" Jul 02 19:15:16.689690 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:16.689659 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h7dk4"] Jul 02 19:15:16.692945 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:16.692927 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7dk4" Jul 02 19:15:16.695613 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:16.695590 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-jlwdh\"" Jul 02 19:15:16.711210 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:16.711186 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h7dk4"] Jul 02 19:15:16.785019 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:16.784972 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzflf\" (UniqueName: \"kubernetes.io/projected/c955b662-ccce-46c9-9773-13c92868a7c8-kube-api-access-gzflf\") pod \"authorino-operator-75cc8d5cb7-h7dk4\" (UID: \"c955b662-ccce-46c9-9773-13c92868a7c8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7dk4" Jul 02 19:15:16.885847 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:16.885815 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzflf\" (UniqueName: \"kubernetes.io/projected/c955b662-ccce-46c9-9773-13c92868a7c8-kube-api-access-gzflf\") pod \"authorino-operator-75cc8d5cb7-h7dk4\" (UID: \"c955b662-ccce-46c9-9773-13c92868a7c8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7dk4" Jul 02 19:15:16.897107 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:16.897072 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzflf\" (UniqueName: \"kubernetes.io/projected/c955b662-ccce-46c9-9773-13c92868a7c8-kube-api-access-gzflf\") pod \"authorino-operator-75cc8d5cb7-h7dk4\" (UID: \"c955b662-ccce-46c9-9773-13c92868a7c8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7dk4" Jul 02 19:15:17.003146 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:17.003063 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7dk4" Jul 02 19:15:17.140150 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:17.140121 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h7dk4"] Jul 02 19:15:17.142279 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:15:17.142253 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc955b662_ccce_46c9_9773_13c92868a7c8.slice/crio-70f5744c88b742db6dc89d9376eacaa3d295db37282e8cc3441002146090cc40 WatchSource:0}: Error finding container 70f5744c88b742db6dc89d9376eacaa3d295db37282e8cc3441002146090cc40: Status 404 returned error can't find the container with id 70f5744c88b742db6dc89d9376eacaa3d295db37282e8cc3441002146090cc40 Jul 02 19:15:17.736130 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:17.736098 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7dk4" event={"ID":"c955b662-ccce-46c9-9773-13c92868a7c8","Type":"ContainerStarted","Data":"70f5744c88b742db6dc89d9376eacaa3d295db37282e8cc3441002146090cc40"} Jul 02 19:15:19.747318 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:19.747227 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7dk4" event={"ID":"c955b662-ccce-46c9-9773-13c92868a7c8","Type":"ContainerStarted","Data":"53cee60e6450027b7580e7c260544f5d368d67509526f24187ff3373babda24b"} Jul 02 19:15:19.747318 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:19.747289 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7dk4" Jul 02 19:15:19.769103 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:19.769056 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7dk4" podStartSLOduration=1.475239897 podStartE2EDuration="3.769041857s" totalImagesPullingTime="2.29380196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:15:19.767717478 +0000 UTC m=+462.570532928" watchObservedRunningTime="2026-07-02 19:15:19.769041857 +0000 UTC m=+462.571857308" Jul 02 19:15:21.522449 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:21.522416 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44"] Jul 02 19:15:21.525801 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:21.525785 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:21.528740 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:21.528719 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-m9ktd\"" Jul 02 19:15:21.541090 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:21.541069 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44"] Jul 02 19:15:21.620491 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:21.620463 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk6rv\" (UniqueName: \"kubernetes.io/projected/0f3d831b-90c4-462e-9e06-df969667c276-kube-api-access-vk6rv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4cz44\" (UID: \"0f3d831b-90c4-462e-9e06-df969667c276\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:21.620616 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:21.620498 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f3d831b-90c4-462e-9e06-df969667c276-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4cz44\" (UID: \"0f3d831b-90c4-462e-9e06-df969667c276\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:21.721838 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:21.721817 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk6rv\" (UniqueName: \"kubernetes.io/projected/0f3d831b-90c4-462e-9e06-df969667c276-kube-api-access-vk6rv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4cz44\" (UID: \"0f3d831b-90c4-462e-9e06-df969667c276\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:21.721968 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:21.721847 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f3d831b-90c4-462e-9e06-df969667c276-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4cz44\" (UID: \"0f3d831b-90c4-462e-9e06-df969667c276\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:21.722179 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:21.722163 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f3d831b-90c4-462e-9e06-df969667c276-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4cz44\" (UID: \"0f3d831b-90c4-462e-9e06-df969667c276\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:21.730795 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:21.730767 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk6rv\" (UniqueName: \"kubernetes.io/projected/0f3d831b-90c4-462e-9e06-df969667c276-kube-api-access-vk6rv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4cz44\" (UID: \"0f3d831b-90c4-462e-9e06-df969667c276\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:21.836365 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:21.836341 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:21.971807 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:21.971786 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44"] Jul 02 19:15:21.973774 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:15:21.973746 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f3d831b_90c4_462e_9e06_df969667c276.slice/crio-095516c7149698623a59874640730509cda1baea40180a0f193bc665013e09a0 WatchSource:0}: Error finding container 095516c7149698623a59874640730509cda1baea40180a0f193bc665013e09a0: Status 404 returned error can't find the container with id 095516c7149698623a59874640730509cda1baea40180a0f193bc665013e09a0 Jul 02 19:15:22.761238 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:22.761192 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" event={"ID":"0f3d831b-90c4-462e-9e06-df969667c276","Type":"ContainerStarted","Data":"095516c7149698623a59874640730509cda1baea40180a0f193bc665013e09a0"} Jul 02 19:15:23.712227 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:23.712194 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" Jul 02 19:15:26.733283 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:26.733241 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n9txz" Jul 02 19:15:27.791305 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:27.791268 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" event={"ID":"0f3d831b-90c4-462e-9e06-df969667c276","Type":"ContainerStarted","Data":"8a25a3cc3d9084dea35526ab4bdf83603be7ca8af714ea9f1ce4f8d2393adc86"} Jul 02 19:15:27.791954 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:27.791448 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:27.814464 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:27.814377 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" podStartSLOduration=1.8164696839999999 podStartE2EDuration="6.814361999s" totalImagesPullingTime="4.997892315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:15:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:15:27.813499023 +0000 UTC m=+470.616314494" watchObservedRunningTime="2026-07-02 19:15:27.814361999 +0000 UTC m=+470.617177450" Jul 02 19:15:29.801647 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:29.801613 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8"] Jul 02 19:15:29.805175 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:29.805154 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8" Jul 02 19:15:29.807658 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:29.807633 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 02 19:15:29.807658 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:29.807644 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pfnst\"" Jul 02 19:15:29.807829 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:29.807801 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 02 19:15:29.813466 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:29.813435 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8"] Jul 02 19:15:29.888764 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:29.888734 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdvk2\" (UniqueName: \"kubernetes.io/projected/4e241b2c-f017-4bb6-8fe3-400a3f3ec955-kube-api-access-hdvk2\") pod \"kuadrant-console-plugin-66b5774985-mhxr8\" (UID: \"4e241b2c-f017-4bb6-8fe3-400a3f3ec955\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8" Jul 02 19:15:29.888903 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:29.888769 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4e241b2c-f017-4bb6-8fe3-400a3f3ec955-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mhxr8\" (UID: \"4e241b2c-f017-4bb6-8fe3-400a3f3ec955\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8" Jul 02 19:15:29.888903 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:29.888854 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e241b2c-f017-4bb6-8fe3-400a3f3ec955-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mhxr8\" (UID: \"4e241b2c-f017-4bb6-8fe3-400a3f3ec955\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8" Jul 02 19:15:29.990156 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:29.990129 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e241b2c-f017-4bb6-8fe3-400a3f3ec955-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mhxr8\" (UID: \"4e241b2c-f017-4bb6-8fe3-400a3f3ec955\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8" Jul 02 19:15:29.990275 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:29.990188 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdvk2\" (UniqueName: \"kubernetes.io/projected/4e241b2c-f017-4bb6-8fe3-400a3f3ec955-kube-api-access-hdvk2\") pod \"kuadrant-console-plugin-66b5774985-mhxr8\" (UID: \"4e241b2c-f017-4bb6-8fe3-400a3f3ec955\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8" Jul 02 19:15:29.990275 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:29.990206 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4e241b2c-f017-4bb6-8fe3-400a3f3ec955-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mhxr8\" (UID: \"4e241b2c-f017-4bb6-8fe3-400a3f3ec955\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8" Jul 02 19:15:29.990360 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:29.990268 2578 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 02 19:15:29.990360 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:29.990332 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e241b2c-f017-4bb6-8fe3-400a3f3ec955-plugin-serving-cert podName:4e241b2c-f017-4bb6-8fe3-400a3f3ec955 nodeName:}" failed. No retries permitted until 2026-07-02 19:15:30.490316279 +0000 UTC m=+473.293131707 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/4e241b2c-f017-4bb6-8fe3-400a3f3ec955-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-mhxr8" (UID: "4e241b2c-f017-4bb6-8fe3-400a3f3ec955") : secret "plugin-serving-cert" not found Jul 02 19:15:29.990811 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:29.990794 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4e241b2c-f017-4bb6-8fe3-400a3f3ec955-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mhxr8\" (UID: \"4e241b2c-f017-4bb6-8fe3-400a3f3ec955\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8" Jul 02 19:15:30.001544 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:30.001521 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdvk2\" (UniqueName: \"kubernetes.io/projected/4e241b2c-f017-4bb6-8fe3-400a3f3ec955-kube-api-access-hdvk2\") pod \"kuadrant-console-plugin-66b5774985-mhxr8\" (UID: \"4e241b2c-f017-4bb6-8fe3-400a3f3ec955\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8" Jul 02 19:15:30.492979 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:30.492947 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e241b2c-f017-4bb6-8fe3-400a3f3ec955-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mhxr8\" (UID: \"4e241b2c-f017-4bb6-8fe3-400a3f3ec955\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8" Jul 02 19:15:30.495407 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:30.495369 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e241b2c-f017-4bb6-8fe3-400a3f3ec955-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mhxr8\" (UID: \"4e241b2c-f017-4bb6-8fe3-400a3f3ec955\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8" Jul 02 19:15:30.714886 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:30.714856 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8" Jul 02 19:15:30.754075 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:30.754005 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7dk4" Jul 02 19:15:30.855532 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:30.855500 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8"] Jul 02 19:15:30.858664 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:15:30.858638 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e241b2c_f017_4bb6_8fe3_400a3f3ec955.slice/crio-c1bfe51629cd56bffa0a9e03a2fd12b998df499a45eca5c6700c837d7779b6f4 WatchSource:0}: Error finding container c1bfe51629cd56bffa0a9e03a2fd12b998df499a45eca5c6700c837d7779b6f4: Status 404 returned error can't find the container with id c1bfe51629cd56bffa0a9e03a2fd12b998df499a45eca5c6700c837d7779b6f4 Jul 02 19:15:31.815825 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:31.815783 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8" event={"ID":"4e241b2c-f017-4bb6-8fe3-400a3f3ec955","Type":"ContainerStarted","Data":"c1bfe51629cd56bffa0a9e03a2fd12b998df499a45eca5c6700c837d7779b6f4"} Jul 02 19:15:38.799111 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:38.799079 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:40.195112 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.195076 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d"] Jul 02 19:15:40.198894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.198871 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d" Jul 02 19:15:40.232039 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.232015 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d"] Jul 02 19:15:40.283080 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.283054 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/176fa5a1-ba9f-42b2-bd78-8a6134a2bca1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5gq9d\" (UID: \"176fa5a1-ba9f-42b2-bd78-8a6134a2bca1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d" Jul 02 19:15:40.283193 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.283110 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgx7p\" (UniqueName: \"kubernetes.io/projected/176fa5a1-ba9f-42b2-bd78-8a6134a2bca1-kube-api-access-dgx7p\") pod \"kuadrant-operator-controller-manager-55467d9f7-5gq9d\" (UID: \"176fa5a1-ba9f-42b2-bd78-8a6134a2bca1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d" Jul 02 19:15:40.383972 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.383948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgx7p\" (UniqueName: \"kubernetes.io/projected/176fa5a1-ba9f-42b2-bd78-8a6134a2bca1-kube-api-access-dgx7p\") pod \"kuadrant-operator-controller-manager-55467d9f7-5gq9d\" (UID: \"176fa5a1-ba9f-42b2-bd78-8a6134a2bca1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d" Jul 02 19:15:40.384098 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.384081 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/176fa5a1-ba9f-42b2-bd78-8a6134a2bca1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5gq9d\" (UID: \"176fa5a1-ba9f-42b2-bd78-8a6134a2bca1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d" Jul 02 19:15:40.384428 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.384411 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/176fa5a1-ba9f-42b2-bd78-8a6134a2bca1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5gq9d\" (UID: \"176fa5a1-ba9f-42b2-bd78-8a6134a2bca1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d" Jul 02 19:15:40.393071 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.393045 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgx7p\" (UniqueName: \"kubernetes.io/projected/176fa5a1-ba9f-42b2-bd78-8a6134a2bca1-kube-api-access-dgx7p\") pod \"kuadrant-operator-controller-manager-55467d9f7-5gq9d\" (UID: \"176fa5a1-ba9f-42b2-bd78-8a6134a2bca1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d" Jul 02 19:15:40.511352 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.511284 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d" Jul 02 19:15:40.566218 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.566188 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d"] Jul 02 19:15:40.572707 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.572680 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d"] Jul 02 19:15:40.594549 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.594522 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r"] Jul 02 19:15:40.599056 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.598996 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" Jul 02 19:15:40.608504 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.608078 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44"] Jul 02 19:15:40.608504 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.608349 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" podUID="0f3d831b-90c4-462e-9e06-df969667c276" containerName="manager" containerID="cri-o://8a25a3cc3d9084dea35526ab4bdf83603be7ca8af714ea9f1ce4f8d2393adc86" gracePeriod=2 Jul 02 19:15:40.618629 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.618604 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r"] Jul 02 19:15:40.629250 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.629203 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44"] Jul 02 19:15:40.643082 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.642963 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4cz44\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="0f3d831b-90c4-462e-9e06-df969667c276" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:40.645569 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.645035 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v"] Jul 02 19:15:40.645569 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.645259 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" podUID="6f77cf09-a4ca-484c-bded-4c5775d29e2b" containerName="manager" containerID="cri-o://34dadada5d95b9ad8023888daf89386f13dfe30b75d06cd70137a151f0a24b1b" gracePeriod=2 Jul 02 19:15:40.648032 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.648000 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q6p7v\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="6f77cf09-a4ca-484c-bded-4c5775d29e2b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" Jul 02 19:15:40.650683 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.650651 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4cz44\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="0f3d831b-90c4-462e-9e06-df969667c276" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:40.651378 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.651328 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v"] Jul 02 19:15:40.686531 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.686503 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvk7x"] Jul 02 19:15:40.687116 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.686971 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f77cf09-a4ca-484c-bded-4c5775d29e2b" containerName="manager" Jul 02 19:15:40.687116 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.687009 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f77cf09-a4ca-484c-bded-4c5775d29e2b" containerName="manager" Jul 02 19:15:40.687116 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.687033 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f3d831b-90c4-462e-9e06-df969667c276" containerName="manager" Jul 02 19:15:40.687116 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.687041 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f3d831b-90c4-462e-9e06-df969667c276" containerName="manager" Jul 02 19:15:40.687380 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.687121 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f77cf09-a4ca-484c-bded-4c5775d29e2b" containerName="manager" Jul 02 19:15:40.687380 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.687135 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f3d831b-90c4-462e-9e06-df969667c276" containerName="manager" Jul 02 19:15:40.687636 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.687607 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-z8n9r\" (UID: \"035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" Jul 02 19:15:40.687745 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.687725 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkgwc\" (UniqueName: \"kubernetes.io/projected/035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae-kube-api-access-gkgwc\") pod \"kuadrant-operator-controller-manager-55467d9f7-z8n9r\" (UID: \"035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" Jul 02 19:15:40.690342 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.690321 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvk7x" Jul 02 19:15:40.692733 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.692708 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q6p7v\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="6f77cf09-a4ca-484c-bded-4c5775d29e2b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" Jul 02 19:15:40.694892 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.694866 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4cz44\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="0f3d831b-90c4-462e-9e06-df969667c276" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:40.701815 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.701798 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvk7x"] Jul 02 19:15:40.788573 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.788541 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkgwc\" (UniqueName: \"kubernetes.io/projected/035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae-kube-api-access-gkgwc\") pod \"kuadrant-operator-controller-manager-55467d9f7-z8n9r\" (UID: \"035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" Jul 02 19:15:40.788753 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.788604 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgr4h\" (UniqueName: \"kubernetes.io/projected/ef79892f-04af-465b-9a83-aa15d89925dc-kube-api-access-jgr4h\") pod \"limitador-operator-controller-manager-859d7b78bd-xvk7x\" (UID: \"ef79892f-04af-465b-9a83-aa15d89925dc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvk7x" Jul 02 19:15:40.788753 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.788703 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-z8n9r\" (UID: \"035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" Jul 02 19:15:40.789194 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.789168 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-z8n9r\" (UID: \"035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" Jul 02 19:15:40.803869 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.803433 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkgwc\" (UniqueName: \"kubernetes.io/projected/035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae-kube-api-access-gkgwc\") pod \"kuadrant-operator-controller-manager-55467d9f7-z8n9r\" (UID: \"035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" Jul 02 19:15:40.814268 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.814231 2578 log.go:32] "RunPodSandbox from runtime service failed" err=< Jul 02 19:15:40.814268 ip-10-0-133-161 kubenswrapper[2578]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-5gq9d_kuadrant-system_176fa5a1-ba9f-42b2-bd78-8a6134a2bca1_0(17e14546a22bfb6bd048df308aa483542dcd33c81df3431443e84892cd3d8c51): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-5gq9d to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"17e14546a22bfb6bd048df308aa483542dcd33c81df3431443e84892cd3d8c51" Netns:"/var/run/netns/390ae5a6-6d27-4833-8bec-91e38cb45b4f" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-5gq9d;K8S_POD_INFRA_CONTAINER_ID=17e14546a22bfb6bd048df308aa483542dcd33c81df3431443e84892cd3d8c51;K8S_POD_UID=176fa5a1-ba9f-42b2-bd78-8a6134a2bca1" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d/176fa5a1-ba9f-42b2-bd78-8a6134a2bca1:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d 17e14546a22bfb6bd048df308aa483542dcd33c81df3431443e84892cd3d8c51 network default NAD default NAD key default] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d 17e14546a22bfb6bd048df308aa483542dcd33c81df3431443e84892cd3d8c51 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:84:00:27 [10.132.0.39/23] Jul 02 19:15:40.814268 ip-10-0-133-161 kubenswrapper[2578]: ' Jul 02 19:15:40.814268 ip-10-0-133-161 kubenswrapper[2578]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jul 02 19:15:40.814268 ip-10-0-133-161 kubenswrapper[2578]: > Jul 02 19:15:40.814527 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.814309 2578 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=< Jul 02 19:15:40.814527 ip-10-0-133-161 kubenswrapper[2578]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-5gq9d_kuadrant-system_176fa5a1-ba9f-42b2-bd78-8a6134a2bca1_0(17e14546a22bfb6bd048df308aa483542dcd33c81df3431443e84892cd3d8c51): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-5gq9d to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"17e14546a22bfb6bd048df308aa483542dcd33c81df3431443e84892cd3d8c51" Netns:"/var/run/netns/390ae5a6-6d27-4833-8bec-91e38cb45b4f" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-5gq9d;K8S_POD_INFRA_CONTAINER_ID=17e14546a22bfb6bd048df308aa483542dcd33c81df3431443e84892cd3d8c51;K8S_POD_UID=176fa5a1-ba9f-42b2-bd78-8a6134a2bca1" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d/176fa5a1-ba9f-42b2-bd78-8a6134a2bca1:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d 17e14546a22bfb6bd048df308aa483542dcd33c81df3431443e84892cd3d8c51 network default NAD default NAD key default] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d 17e14546a22bfb6bd048df308aa483542dcd33c81df3431443e84892cd3d8c51 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:84:00:27 [10.132.0.39/23] Jul 02 19:15:40.814527 ip-10-0-133-161 kubenswrapper[2578]: ' Jul 02 19:15:40.814527 ip-10-0-133-161 kubenswrapper[2578]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jul 02 19:15:40.814527 ip-10-0-133-161 kubenswrapper[2578]: > pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d" Jul 02 19:15:40.876068 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.876028 2578 generic.go:356] "Generic (PLEG): container finished" podID="6f77cf09-a4ca-484c-bded-4c5775d29e2b" containerID="34dadada5d95b9ad8023888daf89386f13dfe30b75d06cd70137a151f0a24b1b" exitCode=0 Jul 02 19:15:40.878266 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.878237 2578 generic.go:356] "Generic (PLEG): container finished" podID="0f3d831b-90c4-462e-9e06-df969667c276" containerID="8a25a3cc3d9084dea35526ab4bdf83603be7ca8af714ea9f1ce4f8d2393adc86" exitCode=0 Jul 02 19:15:40.878475 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.878455 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d" Jul 02 19:15:40.880795 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.880760 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4cz44\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="0f3d831b-90c4-462e-9e06-df969667c276" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:40.883437 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.883380 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q6p7v\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="6f77cf09-a4ca-484c-bded-4c5775d29e2b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" Jul 02 19:15:40.885315 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.885296 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:40.886026 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.886004 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5gq9d\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="176fa5a1-ba9f-42b2-bd78-8a6134a2bca1" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d" Jul 02 19:15:40.887348 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.887333 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d" Jul 02 19:15:40.889370 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.889345 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5gq9d\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="176fa5a1-ba9f-42b2-bd78-8a6134a2bca1" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d" Jul 02 19:15:40.889567 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.889551 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgr4h\" (UniqueName: \"kubernetes.io/projected/ef79892f-04af-465b-9a83-aa15d89925dc-kube-api-access-jgr4h\") pod \"limitador-operator-controller-manager-859d7b78bd-xvk7x\" (UID: \"ef79892f-04af-465b-9a83-aa15d89925dc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvk7x" Jul 02 19:15:40.892379 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.892360 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4cz44\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="0f3d831b-90c4-462e-9e06-df969667c276" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:40.896445 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.896421 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q6p7v\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="6f77cf09-a4ca-484c-bded-4c5775d29e2b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" Jul 02 19:15:40.898999 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.898979 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q6p7v\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="6f77cf09-a4ca-484c-bded-4c5775d29e2b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" Jul 02 19:15:40.903817 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.903788 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5gq9d\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="176fa5a1-ba9f-42b2-bd78-8a6134a2bca1" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d" Jul 02 19:15:40.905672 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.905649 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4cz44\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="0f3d831b-90c4-462e-9e06-df969667c276" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:40.907556 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.907530 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgr4h\" (UniqueName: \"kubernetes.io/projected/ef79892f-04af-465b-9a83-aa15d89925dc-kube-api-access-jgr4h\") pod \"limitador-operator-controller-manager-859d7b78bd-xvk7x\" (UID: \"ef79892f-04af-465b-9a83-aa15d89925dc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvk7x" Jul 02 19:15:40.913130 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.913110 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" Jul 02 19:15:40.917857 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.917840 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" Jul 02 19:15:40.920091 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.920070 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5gq9d\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="176fa5a1-ba9f-42b2-bd78-8a6134a2bca1" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d" Jul 02 19:15:40.923473 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.923302 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4cz44\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="0f3d831b-90c4-462e-9e06-df969667c276" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:40.925231 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:40.925198 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q6p7v\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="6f77cf09-a4ca-484c-bded-4c5775d29e2b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" Jul 02 19:15:40.990832 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.990806 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgx7p\" (UniqueName: \"kubernetes.io/projected/176fa5a1-ba9f-42b2-bd78-8a6134a2bca1-kube-api-access-dgx7p\") pod \"176fa5a1-ba9f-42b2-bd78-8a6134a2bca1\" (UID: \"176fa5a1-ba9f-42b2-bd78-8a6134a2bca1\") " Jul 02 19:15:40.990993 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.990851 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vk6rv\" (UniqueName: \"kubernetes.io/projected/0f3d831b-90c4-462e-9e06-df969667c276-kube-api-access-vk6rv\") pod \"0f3d831b-90c4-462e-9e06-df969667c276\" (UID: \"0f3d831b-90c4-462e-9e06-df969667c276\") " Jul 02 19:15:40.990993 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.990891 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs68t\" (UniqueName: \"kubernetes.io/projected/6f77cf09-a4ca-484c-bded-4c5775d29e2b-kube-api-access-bs68t\") pod \"6f77cf09-a4ca-484c-bded-4c5775d29e2b\" (UID: \"6f77cf09-a4ca-484c-bded-4c5775d29e2b\") " Jul 02 19:15:40.990993 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.990950 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/176fa5a1-ba9f-42b2-bd78-8a6134a2bca1-extensions-socket-volume\") pod \"176fa5a1-ba9f-42b2-bd78-8a6134a2bca1\" (UID: \"176fa5a1-ba9f-42b2-bd78-8a6134a2bca1\") " Jul 02 19:15:40.990993 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.990972 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f3d831b-90c4-462e-9e06-df969667c276-extensions-socket-volume\") pod \"0f3d831b-90c4-462e-9e06-df969667c276\" (UID: \"0f3d831b-90c4-462e-9e06-df969667c276\") " Jul 02 19:15:40.991688 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.991636 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/176fa5a1-ba9f-42b2-bd78-8a6134a2bca1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "176fa5a1-ba9f-42b2-bd78-8a6134a2bca1" (UID: "176fa5a1-ba9f-42b2-bd78-8a6134a2bca1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:15:40.991830 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.991700 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f3d831b-90c4-462e-9e06-df969667c276-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0f3d831b-90c4-462e-9e06-df969667c276" (UID: "0f3d831b-90c4-462e-9e06-df969667c276"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:15:40.993788 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.993759 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f3d831b-90c4-462e-9e06-df969667c276-kube-api-access-vk6rv" (OuterVolumeSpecName: "kube-api-access-vk6rv") pod "0f3d831b-90c4-462e-9e06-df969667c276" (UID: "0f3d831b-90c4-462e-9e06-df969667c276"). InnerVolumeSpecName "kube-api-access-vk6rv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:15:40.993965 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.993942 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f77cf09-a4ca-484c-bded-4c5775d29e2b-kube-api-access-bs68t" (OuterVolumeSpecName: "kube-api-access-bs68t") pod "6f77cf09-a4ca-484c-bded-4c5775d29e2b" (UID: "6f77cf09-a4ca-484c-bded-4c5775d29e2b"). InnerVolumeSpecName "kube-api-access-bs68t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:15:40.994351 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:40.994327 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/176fa5a1-ba9f-42b2-bd78-8a6134a2bca1-kube-api-access-dgx7p" (OuterVolumeSpecName: "kube-api-access-dgx7p") pod "176fa5a1-ba9f-42b2-bd78-8a6134a2bca1" (UID: "176fa5a1-ba9f-42b2-bd78-8a6134a2bca1"). InnerVolumeSpecName "kube-api-access-dgx7p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:15:41.019746 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:41.019689 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvk7x" Jul 02 19:15:41.073106 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:41.073084 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r"] Jul 02 19:15:41.091777 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:41.091744 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/176fa5a1-ba9f-42b2-bd78-8a6134a2bca1-extensions-socket-volume\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:15:41.091777 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:41.091777 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f3d831b-90c4-462e-9e06-df969667c276-extensions-socket-volume\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:15:41.091954 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:41.091791 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgx7p\" (UniqueName: \"kubernetes.io/projected/176fa5a1-ba9f-42b2-bd78-8a6134a2bca1-kube-api-access-dgx7p\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:15:41.091954 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:41.091805 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vk6rv\" (UniqueName: \"kubernetes.io/projected/0f3d831b-90c4-462e-9e06-df969667c276-kube-api-access-vk6rv\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:15:41.091954 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:41.091818 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs68t\" (UniqueName: \"kubernetes.io/projected/6f77cf09-a4ca-484c-bded-4c5775d29e2b-kube-api-access-bs68t\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:15:41.728675 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:41.728600 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f3d831b-90c4-462e-9e06-df969667c276" path="/var/lib/kubelet/pods/0f3d831b-90c4-462e-9e06-df969667c276/volumes" Jul 02 19:15:41.729130 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:41.729114 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="176fa5a1-ba9f-42b2-bd78-8a6134a2bca1" path="/var/lib/kubelet/pods/176fa5a1-ba9f-42b2-bd78-8a6134a2bca1/volumes" Jul 02 19:15:41.729410 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:41.729381 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f77cf09-a4ca-484c-bded-4c5775d29e2b" path="/var/lib/kubelet/pods/6f77cf09-a4ca-484c-bded-4c5775d29e2b/volumes" Jul 02 19:15:41.885650 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:41.885619 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6p7v" Jul 02 19:15:41.885811 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:41.885624 2578 scope.go:117] "RemoveContainer" containerID="34dadada5d95b9ad8023888daf89386f13dfe30b75d06cd70137a151f0a24b1b" Jul 02 19:15:41.887729 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:41.887631 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cz44" Jul 02 19:15:41.887729 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:41.887714 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d" Jul 02 19:15:41.895280 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:15:41.895253 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5gq9d\" is forbidden: User \"system:node:ip-10-0-133-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-161.ec2.internal' and this object" podUID="176fa5a1-ba9f-42b2-bd78-8a6134a2bca1" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5gq9d" Jul 02 19:15:54.829010 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:15:54.828967 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod035e34f2_f8d5_4ce0_aa77_dc7f8a7515ae.slice/crio-8834b9dbf54d57907c9ed8cec3f7e85de9c993cd41031fe2503db54cd9f6ae7c WatchSource:0}: Error finding container 8834b9dbf54d57907c9ed8cec3f7e85de9c993cd41031fe2503db54cd9f6ae7c: Status 404 returned error can't find the container with id 8834b9dbf54d57907c9ed8cec3f7e85de9c993cd41031fe2503db54cd9f6ae7c Jul 02 19:15:54.831716 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:54.831697 2578 scope.go:117] "RemoveContainer" containerID="8a25a3cc3d9084dea35526ab4bdf83603be7ca8af714ea9f1ce4f8d2393adc86" Jul 02 19:15:54.961291 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:54.961264 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" event={"ID":"035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae","Type":"ContainerStarted","Data":"8a6ab8176b3a8a9ccf94a2301036bdbefe2544c01e009ef496676075cf648da1"} Jul 02 19:15:54.961452 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:54.961321 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" event={"ID":"035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae","Type":"ContainerStarted","Data":"8834b9dbf54d57907c9ed8cec3f7e85de9c993cd41031fe2503db54cd9f6ae7c"} Jul 02 19:15:54.961452 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:54.961346 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" Jul 02 19:15:54.979560 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:54.979540 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvk7x"] Jul 02 19:15:54.982697 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:15:54.982672 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef79892f_04af_465b_9a83_aa15d89925dc.slice/crio-1d5f3cf509c8325a70558b2e7bb3d737badeb707d8f1e3e7cfe36c016430a413 WatchSource:0}: Error finding container 1d5f3cf509c8325a70558b2e7bb3d737badeb707d8f1e3e7cfe36c016430a413: Status 404 returned error can't find the container with id 1d5f3cf509c8325a70558b2e7bb3d737badeb707d8f1e3e7cfe36c016430a413 Jul 02 19:15:54.991762 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:54.991710 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" podStartSLOduration=14.991690588 podStartE2EDuration="14.991690588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:15:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:15:54.988378495 +0000 UTC m=+497.791193961" watchObservedRunningTime="2026-07-02 19:15:54.991690588 +0000 UTC m=+497.794506041" Jul 02 19:15:55.975213 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:55.975168 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvk7x" event={"ID":"ef79892f-04af-465b-9a83-aa15d89925dc","Type":"ContainerStarted","Data":"15cffbffc3890abe0ccde7da3b70259ffc3905b433203fd7f3096259bfe51165"} Jul 02 19:15:55.975649 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:55.975265 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvk7x" event={"ID":"ef79892f-04af-465b-9a83-aa15d89925dc","Type":"ContainerStarted","Data":"1d5f3cf509c8325a70558b2e7bb3d737badeb707d8f1e3e7cfe36c016430a413"} Jul 02 19:15:55.975649 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:55.975300 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvk7x" Jul 02 19:15:55.976725 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:55.976702 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8" event={"ID":"4e241b2c-f017-4bb6-8fe3-400a3f3ec955","Type":"ContainerStarted","Data":"3aec80e4e4ca4375386a45c10f59c1d2cd3e6af7faec07ea93d7068985b8784b"} Jul 02 19:15:55.995150 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:55.995110 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvk7x" podStartSLOduration=15.995098447 podStartE2EDuration="15.995098447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:15:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:15:55.993922574 +0000 UTC m=+498.796738041" watchObservedRunningTime="2026-07-02 19:15:55.995098447 +0000 UTC m=+498.797913898" Jul 02 19:15:56.011751 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:15:56.011711 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mhxr8" podStartSLOduration=2.942203818 podStartE2EDuration="27.011698711s" totalImagesPullingTime="24.069494893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:15:56.010969958 +0000 UTC m=+498.813785411" watchObservedRunningTime="2026-07-02 19:15:56.011698711 +0000 UTC m=+498.814514201" Jul 02 19:16:05.978945 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:05.978914 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" Jul 02 19:16:06.276800 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:06.276775 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk"] Jul 02 19:16:06.291810 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:06.291786 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" Jul 02 19:16:06.294047 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:06.294021 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnh4t\" (UniqueName: \"kubernetes.io/projected/9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf-kube-api-access-pnh4t\") pod \"kuadrant-operator-controller-manager-86878c769f-mwmdk\" (UID: \"9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" Jul 02 19:16:06.294174 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:06.294096 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mwmdk\" (UID: \"9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" Jul 02 19:16:06.295988 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:06.295968 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk"] Jul 02 19:16:06.394577 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:06.394545 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mwmdk\" (UID: \"9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" Jul 02 19:16:06.394749 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:06.394601 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnh4t\" (UniqueName: \"kubernetes.io/projected/9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf-kube-api-access-pnh4t\") pod \"kuadrant-operator-controller-manager-86878c769f-mwmdk\" (UID: \"9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" Jul 02 19:16:06.394950 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:06.394932 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mwmdk\" (UID: \"9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" Jul 02 19:16:06.407232 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:06.407208 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnh4t\" (UniqueName: \"kubernetes.io/projected/9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf-kube-api-access-pnh4t\") pod \"kuadrant-operator-controller-manager-86878c769f-mwmdk\" (UID: \"9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" Jul 02 19:16:06.604147 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:06.604115 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" Jul 02 19:16:06.744938 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:06.744803 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk"] Jul 02 19:16:06.747720 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:16:06.747695 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a457d1c_ce3a_48d3_8d7b_db4eb34d75cf.slice/crio-58e260084486367407729afb4641797e42865982520f04281ca41546fb1b8fbe WatchSource:0}: Error finding container 58e260084486367407729afb4641797e42865982520f04281ca41546fb1b8fbe: Status 404 returned error can't find the container with id 58e260084486367407729afb4641797e42865982520f04281ca41546fb1b8fbe Jul 02 19:16:06.983263 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:06.983185 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvk7x" Jul 02 19:16:07.031959 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:07.031926 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" event={"ID":"9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf","Type":"ContainerStarted","Data":"0878a472ce7db50da307f37da6373c0a0a0ffa51e4c96a973d5bfc136c535f07"} Jul 02 19:16:07.032118 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:07.031981 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" event={"ID":"9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf","Type":"ContainerStarted","Data":"58e260084486367407729afb4641797e42865982520f04281ca41546fb1b8fbe"} Jul 02 19:16:07.032118 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:07.032100 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" Jul 02 19:16:07.052041 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:07.051998 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" podStartSLOduration=1.051982747 podStartE2EDuration="1.051982747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:16:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:16:07.051167797 +0000 UTC m=+509.853983268" watchObservedRunningTime="2026-07-02 19:16:07.051982747 +0000 UTC m=+509.854798196" Jul 02 19:16:18.039747 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:18.039717 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" Jul 02 19:16:18.093280 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:18.093251 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r"] Jul 02 19:16:18.093477 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:18.093442 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" podUID="035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae" containerName="manager" containerID="cri-o://8a6ab8176b3a8a9ccf94a2301036bdbefe2544c01e009ef496676075cf648da1" gracePeriod=10 Jul 02 19:16:18.858472 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:18.858446 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" Jul 02 19:16:18.897698 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:18.897674 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae-extensions-socket-volume\") pod \"035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae\" (UID: \"035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae\") " Jul 02 19:16:18.897817 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:18.897727 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkgwc\" (UniqueName: \"kubernetes.io/projected/035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae-kube-api-access-gkgwc\") pod \"035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae\" (UID: \"035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae\") " Jul 02 19:16:18.898114 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:18.898089 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae" (UID: "035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:16:18.899949 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:18.899925 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae-kube-api-access-gkgwc" (OuterVolumeSpecName: "kube-api-access-gkgwc") pod "035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae" (UID: "035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae"). InnerVolumeSpecName "kube-api-access-gkgwc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:16:18.998799 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:18.998745 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae-extensions-socket-volume\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:16:18.998799 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:18.998768 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkgwc\" (UniqueName: \"kubernetes.io/projected/035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae-kube-api-access-gkgwc\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:16:19.097285 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:19.097257 2578 generic.go:356] "Generic (PLEG): container finished" podID="035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae" containerID="8a6ab8176b3a8a9ccf94a2301036bdbefe2544c01e009ef496676075cf648da1" exitCode=0 Jul 02 19:16:19.097711 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:19.097305 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" Jul 02 19:16:19.097711 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:19.097346 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" event={"ID":"035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae","Type":"ContainerDied","Data":"8a6ab8176b3a8a9ccf94a2301036bdbefe2544c01e009ef496676075cf648da1"} Jul 02 19:16:19.097711 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:19.097417 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r" event={"ID":"035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae","Type":"ContainerDied","Data":"8834b9dbf54d57907c9ed8cec3f7e85de9c993cd41031fe2503db54cd9f6ae7c"} Jul 02 19:16:19.097711 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:19.097442 2578 scope.go:117] "RemoveContainer" containerID="8a6ab8176b3a8a9ccf94a2301036bdbefe2544c01e009ef496676075cf648da1" Jul 02 19:16:19.106064 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:19.106048 2578 scope.go:117] "RemoveContainer" containerID="8a6ab8176b3a8a9ccf94a2301036bdbefe2544c01e009ef496676075cf648da1" Jul 02 19:16:19.106301 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:16:19.106284 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a6ab8176b3a8a9ccf94a2301036bdbefe2544c01e009ef496676075cf648da1\": container with ID starting with 8a6ab8176b3a8a9ccf94a2301036bdbefe2544c01e009ef496676075cf648da1 not found: ID does not exist" containerID="8a6ab8176b3a8a9ccf94a2301036bdbefe2544c01e009ef496676075cf648da1" Jul 02 19:16:19.106346 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:19.106313 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a6ab8176b3a8a9ccf94a2301036bdbefe2544c01e009ef496676075cf648da1"} err="failed to get container status \"8a6ab8176b3a8a9ccf94a2301036bdbefe2544c01e009ef496676075cf648da1\": rpc error: code = NotFound desc = could not find container \"8a6ab8176b3a8a9ccf94a2301036bdbefe2544c01e009ef496676075cf648da1\": container with ID starting with 8a6ab8176b3a8a9ccf94a2301036bdbefe2544c01e009ef496676075cf648da1 not found: ID does not exist" Jul 02 19:16:19.124206 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:19.124186 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r"] Jul 02 19:16:19.131314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:19.131297 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z8n9r"] Jul 02 19:16:19.727744 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:19.727711 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae" path="/var/lib/kubelet/pods/035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae/volumes" Jul 02 19:16:22.414548 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.414518 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6"] Jul 02 19:16:22.414896 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.414874 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae" containerName="manager" Jul 02 19:16:22.414896 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.414887 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae" containerName="manager" Jul 02 19:16:22.414965 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.414940 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="035e34f2-f8d5-4ce0-aa77-dc7f8a7515ae" containerName="manager" Jul 02 19:16:22.432871 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.432843 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6"] Jul 02 19:16:22.433010 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.432937 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.435421 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.435387 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-fprq4\"" Jul 02 19:16:22.527832 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.527802 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d305985-0721-40bc-85bc-bcd412ab870c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.527976 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.527839 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d305985-0721-40bc-85bc-bcd412ab870c-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.527976 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.527867 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d305985-0721-40bc-85bc-bcd412ab870c-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.527976 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.527912 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb6xs\" (UniqueName: \"kubernetes.io/projected/5d305985-0721-40bc-85bc-bcd412ab870c-kube-api-access-zb6xs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.527976 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.527948 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d305985-0721-40bc-85bc-bcd412ab870c-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.528192 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.527988 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d305985-0721-40bc-85bc-bcd412ab870c-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.528192 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.528028 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d305985-0721-40bc-85bc-bcd412ab870c-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.528192 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.528051 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d305985-0721-40bc-85bc-bcd412ab870c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.528192 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.528099 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d305985-0721-40bc-85bc-bcd412ab870c-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.629449 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.629391 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d305985-0721-40bc-85bc-bcd412ab870c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.629449 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.629454 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d305985-0721-40bc-85bc-bcd412ab870c-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.629663 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.629475 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d305985-0721-40bc-85bc-bcd412ab870c-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.629663 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.629495 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb6xs\" (UniqueName: \"kubernetes.io/projected/5d305985-0721-40bc-85bc-bcd412ab870c-kube-api-access-zb6xs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.629663 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.629513 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d305985-0721-40bc-85bc-bcd412ab870c-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.629663 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.629606 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d305985-0721-40bc-85bc-bcd412ab870c-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.629876 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.629669 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d305985-0721-40bc-85bc-bcd412ab870c-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.629876 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.629692 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d305985-0721-40bc-85bc-bcd412ab870c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.629876 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.629740 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d305985-0721-40bc-85bc-bcd412ab870c-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.630028 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.629911 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d305985-0721-40bc-85bc-bcd412ab870c-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.630028 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.629966 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d305985-0721-40bc-85bc-bcd412ab870c-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.630204 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.630184 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d305985-0721-40bc-85bc-bcd412ab870c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.630263 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.630203 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d305985-0721-40bc-85bc-bcd412ab870c-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.630263 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.630245 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d305985-0721-40bc-85bc-bcd412ab870c-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.631925 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.631908 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d305985-0721-40bc-85bc-bcd412ab870c-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.632002 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.631984 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d305985-0721-40bc-85bc-bcd412ab870c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.638047 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.638023 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d305985-0721-40bc-85bc-bcd412ab870c-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.638257 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.638237 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb6xs\" (UniqueName: \"kubernetes.io/projected/5d305985-0721-40bc-85bc-bcd412ab870c-kube-api-access-zb6xs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-n2ds6\" (UID: \"5d305985-0721-40bc-85bc-bcd412ab870c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.743234 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.743164 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:22.875863 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.875833 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6"] Jul 02 19:16:22.881973 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:16:22.881899 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d305985_0721_40bc_85bc_bcd412ab870c.slice/crio-7518f5a6011b6ca0ec0a5361332714e971ad2ceed8981b19701c6367c97cda57 WatchSource:0}: Error finding container 7518f5a6011b6ca0ec0a5361332714e971ad2ceed8981b19701c6367c97cda57: Status 404 returned error can't find the container with id 7518f5a6011b6ca0ec0a5361332714e971ad2ceed8981b19701c6367c97cda57 Jul 02 19:16:22.883529 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.883493 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:16:22.883606 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.883565 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:16:22.883606 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:22.883596 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:16:23.122422 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:23.122364 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" event={"ID":"5d305985-0721-40bc-85bc-bcd412ab870c","Type":"ContainerStarted","Data":"ab8336b51e40a8da53e81afc9cce5a0d4e2a079a3c5a9f030b27930abd70191b"} Jul 02 19:16:23.122611 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:23.122447 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" event={"ID":"5d305985-0721-40bc-85bc-bcd412ab870c","Type":"ContainerStarted","Data":"7518f5a6011b6ca0ec0a5361332714e971ad2ceed8981b19701c6367c97cda57"} Jul 02 19:16:23.146186 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:23.146137 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" podStartSLOduration=1.146121965 podStartE2EDuration="1.146121965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:16:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:16:23.144208338 +0000 UTC m=+525.947023810" watchObservedRunningTime="2026-07-02 19:16:23.146121965 +0000 UTC m=+525.948937415" Jul 02 19:16:23.744184 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:23.744150 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:23.749127 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:23.749090 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:24.128361 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:24.128330 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:24.129086 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:24.129070 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:16:26.310143 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:26.310109 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-p9zgr"] Jul 02 19:16:26.471138 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:26.471100 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-p9zgr"] Jul 02 19:16:26.471290 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:26.471186 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-p9zgr" Jul 02 19:16:26.473624 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:26.473601 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7zcj4\"" Jul 02 19:16:26.492286 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:26.492265 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-8kvzg"] Jul 02 19:16:26.506572 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:26.506551 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8kvzg"] Jul 02 19:16:26.506669 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:26.506628 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8kvzg" Jul 02 19:16:26.562152 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:26.562087 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg7sw\" (UniqueName: \"kubernetes.io/projected/bc1b88a5-cc63-4a9d-8aac-acbd13f1c316-kube-api-access-gg7sw\") pod \"authorino-58cd955675-p9zgr\" (UID: \"bc1b88a5-cc63-4a9d-8aac-acbd13f1c316\") " pod="kuadrant-system/authorino-58cd955675-p9zgr" Jul 02 19:16:26.562152 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:26.562122 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wppfw\" (UniqueName: \"kubernetes.io/projected/43ba199c-108e-4351-ab1b-05247bcdb785-kube-api-access-wppfw\") pod \"authorino-75d556bd77-8kvzg\" (UID: \"43ba199c-108e-4351-ab1b-05247bcdb785\") " pod="kuadrant-system/authorino-75d556bd77-8kvzg" Jul 02 19:16:26.663596 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:26.663559 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg7sw\" (UniqueName: \"kubernetes.io/projected/bc1b88a5-cc63-4a9d-8aac-acbd13f1c316-kube-api-access-gg7sw\") pod \"authorino-58cd955675-p9zgr\" (UID: \"bc1b88a5-cc63-4a9d-8aac-acbd13f1c316\") " pod="kuadrant-system/authorino-58cd955675-p9zgr" Jul 02 19:16:26.663751 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:26.663605 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wppfw\" (UniqueName: \"kubernetes.io/projected/43ba199c-108e-4351-ab1b-05247bcdb785-kube-api-access-wppfw\") pod \"authorino-75d556bd77-8kvzg\" (UID: \"43ba199c-108e-4351-ab1b-05247bcdb785\") " pod="kuadrant-system/authorino-75d556bd77-8kvzg" Jul 02 19:16:26.672023 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:26.671990 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg7sw\" (UniqueName: \"kubernetes.io/projected/bc1b88a5-cc63-4a9d-8aac-acbd13f1c316-kube-api-access-gg7sw\") pod \"authorino-58cd955675-p9zgr\" (UID: \"bc1b88a5-cc63-4a9d-8aac-acbd13f1c316\") " pod="kuadrant-system/authorino-58cd955675-p9zgr" Jul 02 19:16:26.672131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:26.671990 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wppfw\" (UniqueName: \"kubernetes.io/projected/43ba199c-108e-4351-ab1b-05247bcdb785-kube-api-access-wppfw\") pod \"authorino-75d556bd77-8kvzg\" (UID: \"43ba199c-108e-4351-ab1b-05247bcdb785\") " pod="kuadrant-system/authorino-75d556bd77-8kvzg" Jul 02 19:16:26.781140 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:26.781100 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-p9zgr" Jul 02 19:16:26.816933 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:26.816873 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8kvzg" Jul 02 19:16:27.004298 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:27.004012 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-p9zgr"] Jul 02 19:16:27.006162 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:16:27.006136 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc1b88a5_cc63_4a9d_8aac_acbd13f1c316.slice/crio-d57aec7c65dc90e091fe75d6a39387fc5b8144551090054c87769e3d412840e8 WatchSource:0}: Error finding container d57aec7c65dc90e091fe75d6a39387fc5b8144551090054c87769e3d412840e8: Status 404 returned error can't find the container with id d57aec7c65dc90e091fe75d6a39387fc5b8144551090054c87769e3d412840e8 Jul 02 19:16:27.027520 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:27.027501 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8kvzg"] Jul 02 19:16:27.029232 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:16:27.029210 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43ba199c_108e_4351_ab1b_05247bcdb785.slice/crio-3b957539a90012630fd7dbd3e251f3b2b5c0ef781459888ae98f4cd291ccc2e2 WatchSource:0}: Error finding container 3b957539a90012630fd7dbd3e251f3b2b5c0ef781459888ae98f4cd291ccc2e2: Status 404 returned error can't find the container with id 3b957539a90012630fd7dbd3e251f3b2b5c0ef781459888ae98f4cd291ccc2e2 Jul 02 19:16:27.146071 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:27.145977 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8kvzg" event={"ID":"43ba199c-108e-4351-ab1b-05247bcdb785","Type":"ContainerStarted","Data":"3b957539a90012630fd7dbd3e251f3b2b5c0ef781459888ae98f4cd291ccc2e2"} Jul 02 19:16:27.147071 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:27.147049 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-p9zgr" event={"ID":"bc1b88a5-cc63-4a9d-8aac-acbd13f1c316","Type":"ContainerStarted","Data":"d57aec7c65dc90e091fe75d6a39387fc5b8144551090054c87769e3d412840e8"} Jul 02 19:16:31.173824 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:31.173787 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8kvzg" event={"ID":"43ba199c-108e-4351-ab1b-05247bcdb785","Type":"ContainerStarted","Data":"64953c7619229482f3d90df925963383e4d6f468747173974f2a4657e5667b6c"} Jul 02 19:16:31.175150 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:31.175127 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-p9zgr" event={"ID":"bc1b88a5-cc63-4a9d-8aac-acbd13f1c316","Type":"ContainerStarted","Data":"754cd675f8d62f63bb78450488371af92cc13b1480d0dc5291b096c0fb644aa8"} Jul 02 19:16:31.189936 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:31.189888 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-8kvzg" podStartSLOduration=1.5516107319999999 podStartE2EDuration="5.18987348s" totalImagesPullingTime="3.638262748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:16:31.188533166 +0000 UTC m=+533.991348618" watchObservedRunningTime="2026-07-02 19:16:31.18987348 +0000 UTC m=+533.992688931" Jul 02 19:16:31.206100 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:31.206039 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-p9zgr" podStartSLOduration=1.53337644 podStartE2EDuration="5.206019799s" totalImagesPullingTime="3.672643359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:16:31.202723242 +0000 UTC m=+534.005538696" watchObservedRunningTime="2026-07-02 19:16:31.206019799 +0000 UTC m=+534.008835253" Jul 02 19:16:31.234902 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:31.234870 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-p9zgr"] Jul 02 19:16:33.185644 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:33.185582 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-p9zgr" podUID="bc1b88a5-cc63-4a9d-8aac-acbd13f1c316" containerName="authorino" containerID="cri-o://754cd675f8d62f63bb78450488371af92cc13b1480d0dc5291b096c0fb644aa8" gracePeriod=30 Jul 02 19:16:33.435165 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:33.435137 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-p9zgr" Jul 02 19:16:33.527303 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:33.527212 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gg7sw\" (UniqueName: \"kubernetes.io/projected/bc1b88a5-cc63-4a9d-8aac-acbd13f1c316-kube-api-access-gg7sw\") pod \"bc1b88a5-cc63-4a9d-8aac-acbd13f1c316\" (UID: \"bc1b88a5-cc63-4a9d-8aac-acbd13f1c316\") " Jul 02 19:16:33.529384 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:33.529352 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc1b88a5-cc63-4a9d-8aac-acbd13f1c316-kube-api-access-gg7sw" (OuterVolumeSpecName: "kube-api-access-gg7sw") pod "bc1b88a5-cc63-4a9d-8aac-acbd13f1c316" (UID: "bc1b88a5-cc63-4a9d-8aac-acbd13f1c316"). InnerVolumeSpecName "kube-api-access-gg7sw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:16:33.628438 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:33.628379 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gg7sw\" (UniqueName: \"kubernetes.io/projected/bc1b88a5-cc63-4a9d-8aac-acbd13f1c316-kube-api-access-gg7sw\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:16:34.191601 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:34.191570 2578 generic.go:356] "Generic (PLEG): container finished" podID="bc1b88a5-cc63-4a9d-8aac-acbd13f1c316" containerID="754cd675f8d62f63bb78450488371af92cc13b1480d0dc5291b096c0fb644aa8" exitCode=0 Jul 02 19:16:34.191983 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:34.191611 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-p9zgr" Jul 02 19:16:34.191983 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:34.191656 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-p9zgr" event={"ID":"bc1b88a5-cc63-4a9d-8aac-acbd13f1c316","Type":"ContainerDied","Data":"754cd675f8d62f63bb78450488371af92cc13b1480d0dc5291b096c0fb644aa8"} Jul 02 19:16:34.191983 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:34.191698 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-p9zgr" event={"ID":"bc1b88a5-cc63-4a9d-8aac-acbd13f1c316","Type":"ContainerDied","Data":"d57aec7c65dc90e091fe75d6a39387fc5b8144551090054c87769e3d412840e8"} Jul 02 19:16:34.191983 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:34.191714 2578 scope.go:117] "RemoveContainer" containerID="754cd675f8d62f63bb78450488371af92cc13b1480d0dc5291b096c0fb644aa8" Jul 02 19:16:34.201009 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:34.200989 2578 scope.go:117] "RemoveContainer" containerID="754cd675f8d62f63bb78450488371af92cc13b1480d0dc5291b096c0fb644aa8" Jul 02 19:16:34.201297 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:16:34.201278 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"754cd675f8d62f63bb78450488371af92cc13b1480d0dc5291b096c0fb644aa8\": container with ID starting with 754cd675f8d62f63bb78450488371af92cc13b1480d0dc5291b096c0fb644aa8 not found: ID does not exist" containerID="754cd675f8d62f63bb78450488371af92cc13b1480d0dc5291b096c0fb644aa8" Jul 02 19:16:34.201357 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:34.201304 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"754cd675f8d62f63bb78450488371af92cc13b1480d0dc5291b096c0fb644aa8"} err="failed to get container status \"754cd675f8d62f63bb78450488371af92cc13b1480d0dc5291b096c0fb644aa8\": rpc error: code = NotFound desc = could not find container \"754cd675f8d62f63bb78450488371af92cc13b1480d0dc5291b096c0fb644aa8\": container with ID starting with 754cd675f8d62f63bb78450488371af92cc13b1480d0dc5291b096c0fb644aa8 not found: ID does not exist" Jul 02 19:16:34.209134 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:34.209101 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-p9zgr"] Jul 02 19:16:34.213684 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:34.213665 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-p9zgr"] Jul 02 19:16:35.727214 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:35.727180 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc1b88a5-cc63-4a9d-8aac-acbd13f1c316" path="/var/lib/kubelet/pods/bc1b88a5-cc63-4a9d-8aac-acbd13f1c316/volumes" Jul 02 19:16:57.851098 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:57.851071 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wgb8s"] Jul 02 19:16:57.851632 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:57.851529 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc1b88a5-cc63-4a9d-8aac-acbd13f1c316" containerName="authorino" Jul 02 19:16:57.851632 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:57.851545 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc1b88a5-cc63-4a9d-8aac-acbd13f1c316" containerName="authorino" Jul 02 19:16:57.851632 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:57.851628 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc1b88a5-cc63-4a9d-8aac-acbd13f1c316" containerName="authorino" Jul 02 19:16:57.856183 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:57.856164 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wgb8s" Jul 02 19:16:57.861704 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:57.861684 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wgb8s"] Jul 02 19:16:58.027048 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.027021 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wgb8s"] Jul 02 19:16:58.027268 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:16:58.027250 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-gblmq], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-wgb8s" podUID="81468a61-3448-4f5a-9bad-5b5f816356e8" Jul 02 19:16:58.035075 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.035049 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gblmq\" (UniqueName: \"kubernetes.io/projected/81468a61-3448-4f5a-9bad-5b5f816356e8-kube-api-access-gblmq\") pod \"authorino-5d4b67c446-wgb8s\" (UID: \"81468a61-3448-4f5a-9bad-5b5f816356e8\") " pod="kuadrant-system/authorino-5d4b67c446-wgb8s" Jul 02 19:16:58.054958 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.054930 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-bxx84"] Jul 02 19:16:58.058316 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.058298 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-bxx84" Jul 02 19:16:58.061001 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.060984 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 02 19:16:58.068255 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.068233 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-bxx84"] Jul 02 19:16:58.107500 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.107451 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-bxx84"] Jul 02 19:16:58.107670 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:16:58.107654 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-2xl5q tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-2xl5q tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-bxx84" podUID="89be4a58-909f-4ed3-88f3-25a34bdc45e5" Jul 02 19:16:58.135815 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.135791 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gblmq\" (UniqueName: \"kubernetes.io/projected/81468a61-3448-4f5a-9bad-5b5f816356e8-kube-api-access-gblmq\") pod \"authorino-5d4b67c446-wgb8s\" (UID: \"81468a61-3448-4f5a-9bad-5b5f816356e8\") " pod="kuadrant-system/authorino-5d4b67c446-wgb8s" Jul 02 19:16:58.136578 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.136560 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b998f774d-n7w9w"] Jul 02 19:16:58.139851 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.139833 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b998f774d-n7w9w" Jul 02 19:16:58.147846 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.147828 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b998f774d-n7w9w"] Jul 02 19:16:58.149670 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.149649 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gblmq\" (UniqueName: \"kubernetes.io/projected/81468a61-3448-4f5a-9bad-5b5f816356e8-kube-api-access-gblmq\") pod \"authorino-5d4b67c446-wgb8s\" (UID: \"81468a61-3448-4f5a-9bad-5b5f816356e8\") " pod="kuadrant-system/authorino-5d4b67c446-wgb8s" Jul 02 19:16:58.237071 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.237046 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89be4a58-909f-4ed3-88f3-25a34bdc45e5-tls-cert\") pod \"authorino-76877994-bxx84\" (UID: \"89be4a58-909f-4ed3-88f3-25a34bdc45e5\") " pod="kuadrant-system/authorino-76877994-bxx84" Jul 02 19:16:58.237179 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.237079 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xl5q\" (UniqueName: \"kubernetes.io/projected/89be4a58-909f-4ed3-88f3-25a34bdc45e5-kube-api-access-2xl5q\") pod \"authorino-76877994-bxx84\" (UID: \"89be4a58-909f-4ed3-88f3-25a34bdc45e5\") " pod="kuadrant-system/authorino-76877994-bxx84" Jul 02 19:16:58.237179 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.237131 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2be306f0-83a0-4861-be80-5776d297e0d7-tls-cert\") pod \"authorino-7b998f774d-n7w9w\" (UID: \"2be306f0-83a0-4861-be80-5776d297e0d7\") " pod="kuadrant-system/authorino-7b998f774d-n7w9w" Jul 02 19:16:58.237179 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.237168 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmfvc\" (UniqueName: \"kubernetes.io/projected/2be306f0-83a0-4861-be80-5776d297e0d7-kube-api-access-bmfvc\") pod \"authorino-7b998f774d-n7w9w\" (UID: \"2be306f0-83a0-4861-be80-5776d297e0d7\") " pod="kuadrant-system/authorino-7b998f774d-n7w9w" Jul 02 19:16:58.310758 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.310739 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wgb8s" Jul 02 19:16:58.310897 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.310740 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-bxx84" Jul 02 19:16:58.317785 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.317763 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wgb8s" Jul 02 19:16:58.320910 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.320890 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-bxx84" Jul 02 19:16:58.337944 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.337923 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89be4a58-909f-4ed3-88f3-25a34bdc45e5-tls-cert\") pod \"authorino-76877994-bxx84\" (UID: \"89be4a58-909f-4ed3-88f3-25a34bdc45e5\") " pod="kuadrant-system/authorino-76877994-bxx84" Jul 02 19:16:58.338027 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.337948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xl5q\" (UniqueName: \"kubernetes.io/projected/89be4a58-909f-4ed3-88f3-25a34bdc45e5-kube-api-access-2xl5q\") pod \"authorino-76877994-bxx84\" (UID: \"89be4a58-909f-4ed3-88f3-25a34bdc45e5\") " pod="kuadrant-system/authorino-76877994-bxx84" Jul 02 19:16:58.338027 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.337993 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2be306f0-83a0-4861-be80-5776d297e0d7-tls-cert\") pod \"authorino-7b998f774d-n7w9w\" (UID: \"2be306f0-83a0-4861-be80-5776d297e0d7\") " pod="kuadrant-system/authorino-7b998f774d-n7w9w" Jul 02 19:16:58.338027 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.338023 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmfvc\" (UniqueName: \"kubernetes.io/projected/2be306f0-83a0-4861-be80-5776d297e0d7-kube-api-access-bmfvc\") pod \"authorino-7b998f774d-n7w9w\" (UID: \"2be306f0-83a0-4861-be80-5776d297e0d7\") " pod="kuadrant-system/authorino-7b998f774d-n7w9w" Jul 02 19:16:58.340489 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.340462 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2be306f0-83a0-4861-be80-5776d297e0d7-tls-cert\") pod \"authorino-7b998f774d-n7w9w\" (UID: \"2be306f0-83a0-4861-be80-5776d297e0d7\") " pod="kuadrant-system/authorino-7b998f774d-n7w9w" Jul 02 19:16:58.340820 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.340801 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89be4a58-909f-4ed3-88f3-25a34bdc45e5-tls-cert\") pod \"authorino-76877994-bxx84\" (UID: \"89be4a58-909f-4ed3-88f3-25a34bdc45e5\") " pod="kuadrant-system/authorino-76877994-bxx84" Jul 02 19:16:58.346459 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.346438 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmfvc\" (UniqueName: \"kubernetes.io/projected/2be306f0-83a0-4861-be80-5776d297e0d7-kube-api-access-bmfvc\") pod \"authorino-7b998f774d-n7w9w\" (UID: \"2be306f0-83a0-4861-be80-5776d297e0d7\") " pod="kuadrant-system/authorino-7b998f774d-n7w9w" Jul 02 19:16:58.346533 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.346489 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xl5q\" (UniqueName: \"kubernetes.io/projected/89be4a58-909f-4ed3-88f3-25a34bdc45e5-kube-api-access-2xl5q\") pod \"authorino-76877994-bxx84\" (UID: \"89be4a58-909f-4ed3-88f3-25a34bdc45e5\") " pod="kuadrant-system/authorino-76877994-bxx84" Jul 02 19:16:58.439058 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.439001 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gblmq\" (UniqueName: \"kubernetes.io/projected/81468a61-3448-4f5a-9bad-5b5f816356e8-kube-api-access-gblmq\") pod \"81468a61-3448-4f5a-9bad-5b5f816356e8\" (UID: \"81468a61-3448-4f5a-9bad-5b5f816356e8\") " Jul 02 19:16:58.439058 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.439043 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xl5q\" (UniqueName: \"kubernetes.io/projected/89be4a58-909f-4ed3-88f3-25a34bdc45e5-kube-api-access-2xl5q\") pod \"89be4a58-909f-4ed3-88f3-25a34bdc45e5\" (UID: \"89be4a58-909f-4ed3-88f3-25a34bdc45e5\") " Jul 02 19:16:58.439189 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.439079 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89be4a58-909f-4ed3-88f3-25a34bdc45e5-tls-cert\") pod \"89be4a58-909f-4ed3-88f3-25a34bdc45e5\" (UID: \"89be4a58-909f-4ed3-88f3-25a34bdc45e5\") " Jul 02 19:16:58.440835 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.440805 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81468a61-3448-4f5a-9bad-5b5f816356e8-kube-api-access-gblmq" (OuterVolumeSpecName: "kube-api-access-gblmq") pod "81468a61-3448-4f5a-9bad-5b5f816356e8" (UID: "81468a61-3448-4f5a-9bad-5b5f816356e8"). InnerVolumeSpecName "kube-api-access-gblmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:16:58.440949 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.440931 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89be4a58-909f-4ed3-88f3-25a34bdc45e5-kube-api-access-2xl5q" (OuterVolumeSpecName: "kube-api-access-2xl5q") pod "89be4a58-909f-4ed3-88f3-25a34bdc45e5" (UID: "89be4a58-909f-4ed3-88f3-25a34bdc45e5"). InnerVolumeSpecName "kube-api-access-2xl5q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:16:58.440998 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.440933 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89be4a58-909f-4ed3-88f3-25a34bdc45e5-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "89be4a58-909f-4ed3-88f3-25a34bdc45e5" (UID: "89be4a58-909f-4ed3-88f3-25a34bdc45e5"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:16:58.449106 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.449086 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b998f774d-n7w9w" Jul 02 19:16:58.540290 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.540254 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gblmq\" (UniqueName: \"kubernetes.io/projected/81468a61-3448-4f5a-9bad-5b5f816356e8-kube-api-access-gblmq\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:16:58.540290 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.540291 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xl5q\" (UniqueName: \"kubernetes.io/projected/89be4a58-909f-4ed3-88f3-25a34bdc45e5-kube-api-access-2xl5q\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:16:58.540513 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.540300 2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89be4a58-909f-4ed3-88f3-25a34bdc45e5-tls-cert\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:16:58.581707 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:58.581679 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b998f774d-n7w9w"] Jul 02 19:16:58.584332 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:16:58.584306 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2be306f0_83a0_4861_be80_5776d297e0d7.slice/crio-eb94dc6dcc218049bde82cac354ed6d2df570b507a9a9abc1857aa6970c82776 WatchSource:0}: Error finding container eb94dc6dcc218049bde82cac354ed6d2df570b507a9a9abc1857aa6970c82776: Status 404 returned error can't find the container with id eb94dc6dcc218049bde82cac354ed6d2df570b507a9a9abc1857aa6970c82776 Jul 02 19:16:59.317762 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:59.317735 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-bxx84" Jul 02 19:16:59.318164 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:59.317774 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wgb8s" Jul 02 19:16:59.318164 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:59.317778 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b998f774d-n7w9w" event={"ID":"2be306f0-83a0-4861-be80-5776d297e0d7","Type":"ContainerStarted","Data":"af64a5bb98f3bd529d6ab50367e700612643c597fa6c4e7e1c6cb280f559da77"} Jul 02 19:16:59.318164 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:59.317817 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b998f774d-n7w9w" event={"ID":"2be306f0-83a0-4861-be80-5776d297e0d7","Type":"ContainerStarted","Data":"eb94dc6dcc218049bde82cac354ed6d2df570b507a9a9abc1857aa6970c82776"} Jul 02 19:16:59.346723 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:59.346675 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b998f774d-n7w9w" podStartSLOduration=1.003970763 podStartE2EDuration="1.346661139s" totalImagesPullingTime="342.690376ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:16:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:16:59.344667921 +0000 UTC m=+562.147483373" watchObservedRunningTime="2026-07-02 19:16:59.346661139 +0000 UTC m=+562.149476590" Jul 02 19:16:59.377101 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:59.377077 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wgb8s"] Jul 02 19:16:59.384438 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:59.384409 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8kvzg"] Jul 02 19:16:59.384613 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:59.384592 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-8kvzg" podUID="43ba199c-108e-4351-ab1b-05247bcdb785" containerName="authorino" containerID="cri-o://64953c7619229482f3d90df925963383e4d6f468747173974f2a4657e5667b6c" gracePeriod=30 Jul 02 19:16:59.397184 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:59.397152 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wgb8s"] Jul 02 19:16:59.441282 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:59.441247 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-bxx84"] Jul 02 19:16:59.448604 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:59.448574 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-bxx84"] Jul 02 19:16:59.636025 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:59.635999 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8kvzg" Jul 02 19:16:59.727370 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:59.727342 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81468a61-3448-4f5a-9bad-5b5f816356e8" path="/var/lib/kubelet/pods/81468a61-3448-4f5a-9bad-5b5f816356e8/volumes" Jul 02 19:16:59.727648 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:59.727634 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89be4a58-909f-4ed3-88f3-25a34bdc45e5" path="/var/lib/kubelet/pods/89be4a58-909f-4ed3-88f3-25a34bdc45e5/volumes" Jul 02 19:16:59.749652 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:59.749625 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wppfw\" (UniqueName: \"kubernetes.io/projected/43ba199c-108e-4351-ab1b-05247bcdb785-kube-api-access-wppfw\") pod \"43ba199c-108e-4351-ab1b-05247bcdb785\" (UID: \"43ba199c-108e-4351-ab1b-05247bcdb785\") " Jul 02 19:16:59.751627 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:59.751603 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43ba199c-108e-4351-ab1b-05247bcdb785-kube-api-access-wppfw" (OuterVolumeSpecName: "kube-api-access-wppfw") pod "43ba199c-108e-4351-ab1b-05247bcdb785" (UID: "43ba199c-108e-4351-ab1b-05247bcdb785"). InnerVolumeSpecName "kube-api-access-wppfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:16:59.851207 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:16:59.851181 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wppfw\" (UniqueName: \"kubernetes.io/projected/43ba199c-108e-4351-ab1b-05247bcdb785-kube-api-access-wppfw\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:17:00.324288 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:00.324256 2578 generic.go:356] "Generic (PLEG): container finished" podID="43ba199c-108e-4351-ab1b-05247bcdb785" containerID="64953c7619229482f3d90df925963383e4d6f468747173974f2a4657e5667b6c" exitCode=0 Jul 02 19:17:00.324779 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:00.324293 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8kvzg" Jul 02 19:17:00.324779 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:00.324338 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8kvzg" event={"ID":"43ba199c-108e-4351-ab1b-05247bcdb785","Type":"ContainerDied","Data":"64953c7619229482f3d90df925963383e4d6f468747173974f2a4657e5667b6c"} Jul 02 19:17:00.324779 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:00.324385 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8kvzg" event={"ID":"43ba199c-108e-4351-ab1b-05247bcdb785","Type":"ContainerDied","Data":"3b957539a90012630fd7dbd3e251f3b2b5c0ef781459888ae98f4cd291ccc2e2"} Jul 02 19:17:00.324779 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:00.324422 2578 scope.go:117] "RemoveContainer" containerID="64953c7619229482f3d90df925963383e4d6f468747173974f2a4657e5667b6c" Jul 02 19:17:00.333309 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:00.333292 2578 scope.go:117] "RemoveContainer" containerID="64953c7619229482f3d90df925963383e4d6f468747173974f2a4657e5667b6c" Jul 02 19:17:00.333602 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:17:00.333585 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64953c7619229482f3d90df925963383e4d6f468747173974f2a4657e5667b6c\": container with ID starting with 64953c7619229482f3d90df925963383e4d6f468747173974f2a4657e5667b6c not found: ID does not exist" containerID="64953c7619229482f3d90df925963383e4d6f468747173974f2a4657e5667b6c" Jul 02 19:17:00.333675 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:00.333609 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64953c7619229482f3d90df925963383e4d6f468747173974f2a4657e5667b6c"} err="failed to get container status \"64953c7619229482f3d90df925963383e4d6f468747173974f2a4657e5667b6c\": rpc error: code = NotFound desc = could not find container \"64953c7619229482f3d90df925963383e4d6f468747173974f2a4657e5667b6c\": container with ID starting with 64953c7619229482f3d90df925963383e4d6f468747173974f2a4657e5667b6c not found: ID does not exist" Jul 02 19:17:00.345232 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:00.345210 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8kvzg"] Jul 02 19:17:00.347070 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:00.347053 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8kvzg"] Jul 02 19:17:01.727500 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:01.727466 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43ba199c-108e-4351-ab1b-05247bcdb785" path="/var/lib/kubelet/pods/43ba199c-108e-4351-ab1b-05247bcdb785/volumes" Jul 02 19:17:47.574970 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:47.574938 2578 generic.go:356] "Generic (PLEG): container finished" podID="5d305985-0721-40bc-85bc-bcd412ab870c" containerID="ab8336b51e40a8da53e81afc9cce5a0d4e2a079a3c5a9f030b27930abd70191b" exitCode=0 Jul 02 19:17:47.575415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:47.575019 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" event={"ID":"5d305985-0721-40bc-85bc-bcd412ab870c","Type":"ContainerDied","Data":"ab8336b51e40a8da53e81afc9cce5a0d4e2a079a3c5a9f030b27930abd70191b"} Jul 02 19:17:47.575494 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:47.575475 2578 scope.go:117] "RemoveContainer" containerID="ab8336b51e40a8da53e81afc9cce5a0d4e2a079a3c5a9f030b27930abd70191b" Jul 02 19:17:47.576378 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:47.576351 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:17:47.576497 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:47.576455 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:17:47.576497 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:47.576489 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:17:48.581785 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:48.581751 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" event={"ID":"5d305985-0721-40bc-85bc-bcd412ab870c","Type":"ContainerStarted","Data":"10d142394a6db65efffcae04eb0c235e29fa4ee117fb6499508122db332de45b"} Jul 02 19:17:48.743727 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:48.743686 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:17:48.743981 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:48.743817 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-n2ds6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 02 19:17:48.743981 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:48.743862 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" podUID="5d305985-0721-40bc-85bc-bcd412ab870c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 02 19:17:49.743733 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:49.743694 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-n2ds6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 02 19:17:49.744214 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:49.743767 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" podUID="5d305985-0721-40bc-85bc-bcd412ab870c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 02 19:17:50.744424 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:50.744344 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-n2ds6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 02 19:17:50.744992 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:50.744961 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" podUID="5d305985-0721-40bc-85bc-bcd412ab870c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 02 19:17:51.743669 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:51.743633 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-n2ds6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 02 19:17:51.743935 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:51.743907 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" podUID="5d305985-0721-40bc-85bc-bcd412ab870c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 02 19:17:52.747645 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:52.747615 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:17:52.748059 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:52.747880 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:17:52.748565 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:17:52.748547 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-n2ds6" Jul 02 19:18:26.797175 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.797144 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7"] Jul 02 19:18:26.797758 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.797516 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43ba199c-108e-4351-ab1b-05247bcdb785" containerName="authorino" Jul 02 19:18:26.797758 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.797529 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="43ba199c-108e-4351-ab1b-05247bcdb785" containerName="authorino" Jul 02 19:18:26.797758 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.797610 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="43ba199c-108e-4351-ab1b-05247bcdb785" containerName="authorino" Jul 02 19:18:26.800799 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.800779 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.803581 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.803560 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mvwcw\"" Jul 02 19:18:26.804457 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.804425 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 02 19:18:26.804548 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.804472 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 02 19:18:26.804548 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.804470 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 02 19:18:26.812573 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.812549 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7"] Jul 02 19:18:26.825452 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.825423 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5e2a2d0-a12e-4810-bae1-96e0b166df00-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.825553 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.825464 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phtbn\" (UniqueName: \"kubernetes.io/projected/c5e2a2d0-a12e-4810-bae1-96e0b166df00-kube-api-access-phtbn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.825553 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.825498 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5e2a2d0-a12e-4810-bae1-96e0b166df00-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.825553 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.825536 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5e2a2d0-a12e-4810-bae1-96e0b166df00-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.825679 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.825586 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5e2a2d0-a12e-4810-bae1-96e0b166df00-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.825679 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.825623 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5e2a2d0-a12e-4810-bae1-96e0b166df00-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.925992 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.925961 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5e2a2d0-a12e-4810-bae1-96e0b166df00-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.925992 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.925993 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phtbn\" (UniqueName: \"kubernetes.io/projected/c5e2a2d0-a12e-4810-bae1-96e0b166df00-kube-api-access-phtbn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.926209 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.926011 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5e2a2d0-a12e-4810-bae1-96e0b166df00-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.926209 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.926032 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5e2a2d0-a12e-4810-bae1-96e0b166df00-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.926209 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.926061 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5e2a2d0-a12e-4810-bae1-96e0b166df00-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.926209 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.926083 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5e2a2d0-a12e-4810-bae1-96e0b166df00-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.926486 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.926466 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5e2a2d0-a12e-4810-bae1-96e0b166df00-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.926545 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.926506 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5e2a2d0-a12e-4810-bae1-96e0b166df00-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.926584 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.926549 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5e2a2d0-a12e-4810-bae1-96e0b166df00-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.928461 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.928431 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5e2a2d0-a12e-4810-bae1-96e0b166df00-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.928632 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.928616 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5e2a2d0-a12e-4810-bae1-96e0b166df00-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:26.934346 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:26.934324 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phtbn\" (UniqueName: \"kubernetes.io/projected/c5e2a2d0-a12e-4810-bae1-96e0b166df00-kube-api-access-phtbn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lg2t7\" (UID: \"c5e2a2d0-a12e-4810-bae1-96e0b166df00\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:27.111930 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:27.111900 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:27.247858 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:27.247560 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7"] Jul 02 19:18:27.253160 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:27.253138 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 19:18:27.780631 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:27.780548 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" event={"ID":"c5e2a2d0-a12e-4810-bae1-96e0b166df00","Type":"ContainerStarted","Data":"f2f4d807d7b07a42e6f7cf252e6f5c174c87ed8e50288d5febcf49db3ddfd662"} Jul 02 19:18:28.787259 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:28.787177 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" event={"ID":"c5e2a2d0-a12e-4810-bae1-96e0b166df00","Type":"ContainerStarted","Data":"a45a94b41b2027d04fc871a512a6d668de782fdaf542f58cfad71086734ea3e7"} Jul 02 19:18:28.805938 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:28.805894 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" podStartSLOduration=1.543877377 podStartE2EDuration="2.8058811s" totalImagesPullingTime="1.262003723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:18:28.805083699 +0000 UTC m=+651.607899169" watchObservedRunningTime="2026-07-02 19:18:28.8058811 +0000 UTC m=+651.608696550" Jul 02 19:18:37.112805 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:37.112769 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:37.113173 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:37.112819 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:37.124948 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:37.124924 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:18:37.841179 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:18:37.841156 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lg2t7" Jul 02 19:19:19.314128 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.314098 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb"] Jul 02 19:19:19.317629 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.317607 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.320094 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.320074 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 02 19:19:19.330353 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.330331 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb"] Jul 02 19:19:19.465714 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.465678 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.465873 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.465737 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.465873 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.465815 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.465873 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.465850 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crvf2\" (UniqueName: \"kubernetes.io/projected/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-kube-api-access-crvf2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.465977 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.465877 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.465977 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.465906 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.566295 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.566230 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.566295 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.566279 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.566535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.566310 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.566535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.566329 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crvf2\" (UniqueName: \"kubernetes.io/projected/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-kube-api-access-crvf2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.566535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.566350 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.566535 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.566368 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.566752 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.566710 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.566813 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.566716 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.566873 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.566844 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.568589 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.568570 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.568924 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.568905 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.575262 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.575238 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crvf2\" (UniqueName: \"kubernetes.io/projected/57b4d1f9-ebc3-4988-beb9-ed30f60daae2-kube-api-access-crvf2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb\" (UID: \"57b4d1f9-ebc3-4988-beb9-ed30f60daae2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.629171 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.629145 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:19.764072 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:19.764047 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb"] Jul 02 19:19:19.765815 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:19:19.765793 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57b4d1f9_ebc3_4988_beb9_ed30f60daae2.slice/crio-6447d8ace483cd208bd58fc78fb80fc5e625f81ec362dae529be279dab58322c WatchSource:0}: Error finding container 6447d8ace483cd208bd58fc78fb80fc5e625f81ec362dae529be279dab58322c: Status 404 returned error can't find the container with id 6447d8ace483cd208bd58fc78fb80fc5e625f81ec362dae529be279dab58322c Jul 02 19:19:20.044574 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:20.044548 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" event={"ID":"57b4d1f9-ebc3-4988-beb9-ed30f60daae2","Type":"ContainerStarted","Data":"6447d8ace483cd208bd58fc78fb80fc5e625f81ec362dae529be279dab58322c"} Jul 02 19:19:21.052133 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:21.052095 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" event={"ID":"57b4d1f9-ebc3-4988-beb9-ed30f60daae2","Type":"ContainerStarted","Data":"de44aebc49abd28d9fc7d8f7471e031a2ea5bb598fd3c3e695fc6c7734a575bd"} Jul 02 19:19:21.072544 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:21.072488 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" podStartSLOduration=1.849469385 podStartE2EDuration="2.07246571s" totalImagesPullingTime="222.996325ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:19:21.072339837 +0000 UTC m=+703.875155289" watchObservedRunningTime="2026-07-02 19:19:21.07246571 +0000 UTC m=+703.875281162" Jul 02 19:19:29.629736 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:29.629698 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:29.630344 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:29.629767 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:29.647534 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:29.647504 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:19:30.112164 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:19:30.112085 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb" Jul 02 19:20:23.777916 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:23.777884 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c675c4ddf-trhtp"] Jul 02 19:20:23.781328 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:23.781312 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c675c4ddf-trhtp" Jul 02 19:20:23.791528 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:23.791500 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c675c4ddf-trhtp"] Jul 02 19:20:23.821789 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:23.821758 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/90ccc1f4-459f-49a8-9075-959f9fc619be-tls-cert\") pod \"authorino-6c675c4ddf-trhtp\" (UID: \"90ccc1f4-459f-49a8-9075-959f9fc619be\") " pod="kuadrant-system/authorino-6c675c4ddf-trhtp" Jul 02 19:20:23.821942 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:23.821811 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkdcn\" (UniqueName: \"kubernetes.io/projected/90ccc1f4-459f-49a8-9075-959f9fc619be-kube-api-access-jkdcn\") pod \"authorino-6c675c4ddf-trhtp\" (UID: \"90ccc1f4-459f-49a8-9075-959f9fc619be\") " pod="kuadrant-system/authorino-6c675c4ddf-trhtp" Jul 02 19:20:23.922662 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:23.922632 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkdcn\" (UniqueName: \"kubernetes.io/projected/90ccc1f4-459f-49a8-9075-959f9fc619be-kube-api-access-jkdcn\") pod \"authorino-6c675c4ddf-trhtp\" (UID: \"90ccc1f4-459f-49a8-9075-959f9fc619be\") " pod="kuadrant-system/authorino-6c675c4ddf-trhtp" Jul 02 19:20:23.922790 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:23.922717 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/90ccc1f4-459f-49a8-9075-959f9fc619be-tls-cert\") pod \"authorino-6c675c4ddf-trhtp\" (UID: \"90ccc1f4-459f-49a8-9075-959f9fc619be\") " pod="kuadrant-system/authorino-6c675c4ddf-trhtp" Jul 02 19:20:23.925137 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:23.925115 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/90ccc1f4-459f-49a8-9075-959f9fc619be-tls-cert\") pod \"authorino-6c675c4ddf-trhtp\" (UID: \"90ccc1f4-459f-49a8-9075-959f9fc619be\") " pod="kuadrant-system/authorino-6c675c4ddf-trhtp" Jul 02 19:20:23.931122 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:23.931098 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkdcn\" (UniqueName: \"kubernetes.io/projected/90ccc1f4-459f-49a8-9075-959f9fc619be-kube-api-access-jkdcn\") pod \"authorino-6c675c4ddf-trhtp\" (UID: \"90ccc1f4-459f-49a8-9075-959f9fc619be\") " pod="kuadrant-system/authorino-6c675c4ddf-trhtp" Jul 02 19:20:24.092751 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:24.092720 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c675c4ddf-trhtp" Jul 02 19:20:24.230301 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:24.230271 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c675c4ddf-trhtp"] Jul 02 19:20:24.231757 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:20:24.231729 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90ccc1f4_459f_49a8_9075_959f9fc619be.slice/crio-d6baa4e59b1c762cc57d2ad82aea7407dfc8003c0fe6f980b60a665916fa7925 WatchSource:0}: Error finding container d6baa4e59b1c762cc57d2ad82aea7407dfc8003c0fe6f980b60a665916fa7925: Status 404 returned error can't find the container with id d6baa4e59b1c762cc57d2ad82aea7407dfc8003c0fe6f980b60a665916fa7925 Jul 02 19:20:24.382217 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:24.382133 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c675c4ddf-trhtp" event={"ID":"90ccc1f4-459f-49a8-9075-959f9fc619be","Type":"ContainerStarted","Data":"d6baa4e59b1c762cc57d2ad82aea7407dfc8003c0fe6f980b60a665916fa7925"} Jul 02 19:20:25.391934 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:25.391890 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c675c4ddf-trhtp" event={"ID":"90ccc1f4-459f-49a8-9075-959f9fc619be","Type":"ContainerStarted","Data":"7a6441900e5c4fe13fbb6fca1c7fe4d73804fb4bd3b64c6167d61f473dd5341c"} Jul 02 19:20:25.410493 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:25.410426 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c675c4ddf-trhtp" podStartSLOduration=2.034866833 podStartE2EDuration="2.410388577s" totalImagesPullingTime="375.521744ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:20:25.409567449 +0000 UTC m=+768.212382901" watchObservedRunningTime="2026-07-02 19:20:25.410388577 +0000 UTC m=+768.213204028" Jul 02 19:20:25.442299 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:25.442264 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b998f774d-n7w9w"] Jul 02 19:20:25.442513 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:25.442466 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-7b998f774d-n7w9w" podUID="2be306f0-83a0-4861-be80-5776d297e0d7" containerName="authorino" containerID="cri-o://af64a5bb98f3bd529d6ab50367e700612643c597fa6c4e7e1c6cb280f559da77" gracePeriod=30 Jul 02 19:20:25.708767 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:25.708737 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b998f774d-n7w9w" Jul 02 19:20:25.740280 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:25.739673 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2be306f0-83a0-4861-be80-5776d297e0d7-tls-cert\") pod \"2be306f0-83a0-4861-be80-5776d297e0d7\" (UID: \"2be306f0-83a0-4861-be80-5776d297e0d7\") " Jul 02 19:20:25.740280 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:25.739753 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmfvc\" (UniqueName: \"kubernetes.io/projected/2be306f0-83a0-4861-be80-5776d297e0d7-kube-api-access-bmfvc\") pod \"2be306f0-83a0-4861-be80-5776d297e0d7\" (UID: \"2be306f0-83a0-4861-be80-5776d297e0d7\") " Jul 02 19:20:25.754176 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:25.754134 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2be306f0-83a0-4861-be80-5776d297e0d7-kube-api-access-bmfvc" (OuterVolumeSpecName: "kube-api-access-bmfvc") pod "2be306f0-83a0-4861-be80-5776d297e0d7" (UID: "2be306f0-83a0-4861-be80-5776d297e0d7"). InnerVolumeSpecName "kube-api-access-bmfvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:20:25.763451 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:25.763423 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2be306f0-83a0-4861-be80-5776d297e0d7-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2be306f0-83a0-4861-be80-5776d297e0d7" (UID: "2be306f0-83a0-4861-be80-5776d297e0d7"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:20:25.841307 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:25.841278 2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2be306f0-83a0-4861-be80-5776d297e0d7-tls-cert\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:20:25.841307 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:25.841303 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmfvc\" (UniqueName: \"kubernetes.io/projected/2be306f0-83a0-4861-be80-5776d297e0d7-kube-api-access-bmfvc\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:20:26.398796 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:26.398763 2578 generic.go:356] "Generic (PLEG): container finished" podID="2be306f0-83a0-4861-be80-5776d297e0d7" containerID="af64a5bb98f3bd529d6ab50367e700612643c597fa6c4e7e1c6cb280f559da77" exitCode=0 Jul 02 19:20:26.399234 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:26.398812 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b998f774d-n7w9w" Jul 02 19:20:26.399234 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:26.398843 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b998f774d-n7w9w" event={"ID":"2be306f0-83a0-4861-be80-5776d297e0d7","Type":"ContainerDied","Data":"af64a5bb98f3bd529d6ab50367e700612643c597fa6c4e7e1c6cb280f559da77"} Jul 02 19:20:26.399234 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:26.398889 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b998f774d-n7w9w" event={"ID":"2be306f0-83a0-4861-be80-5776d297e0d7","Type":"ContainerDied","Data":"eb94dc6dcc218049bde82cac354ed6d2df570b507a9a9abc1857aa6970c82776"} Jul 02 19:20:26.399234 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:26.398909 2578 scope.go:117] "RemoveContainer" containerID="af64a5bb98f3bd529d6ab50367e700612643c597fa6c4e7e1c6cb280f559da77" Jul 02 19:20:26.409009 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:26.408980 2578 scope.go:117] "RemoveContainer" containerID="af64a5bb98f3bd529d6ab50367e700612643c597fa6c4e7e1c6cb280f559da77" Jul 02 19:20:26.409287 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:20:26.409270 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af64a5bb98f3bd529d6ab50367e700612643c597fa6c4e7e1c6cb280f559da77\": container with ID starting with af64a5bb98f3bd529d6ab50367e700612643c597fa6c4e7e1c6cb280f559da77 not found: ID does not exist" containerID="af64a5bb98f3bd529d6ab50367e700612643c597fa6c4e7e1c6cb280f559da77" Jul 02 19:20:26.409344 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:26.409296 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af64a5bb98f3bd529d6ab50367e700612643c597fa6c4e7e1c6cb280f559da77"} err="failed to get container status \"af64a5bb98f3bd529d6ab50367e700612643c597fa6c4e7e1c6cb280f559da77\": rpc error: code = NotFound desc = could not find container \"af64a5bb98f3bd529d6ab50367e700612643c597fa6c4e7e1c6cb280f559da77\": container with ID starting with af64a5bb98f3bd529d6ab50367e700612643c597fa6c4e7e1c6cb280f559da77 not found: ID does not exist" Jul 02 19:20:26.443601 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:26.443570 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b998f774d-n7w9w"] Jul 02 19:20:26.450520 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:26.450496 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7b998f774d-n7w9w"] Jul 02 19:20:27.727703 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:20:27.727673 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2be306f0-83a0-4861-be80-5776d297e0d7" path="/var/lib/kubelet/pods/2be306f0-83a0-4861-be80-5776d297e0d7/volumes" Jul 02 19:26:26.214270 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:26.214232 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk"] Jul 02 19:26:26.289248 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:26.289177 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" podUID="9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf" containerName="manager" containerID="cri-o://0878a472ce7db50da307f37da6373c0a0a0ffa51e4c96a973d5bfc136c535f07" gracePeriod=10 Jul 02 19:26:28.940762 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:28.940735 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" Jul 02 19:26:29.051598 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:29.051525 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnh4t\" (UniqueName: \"kubernetes.io/projected/9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf-kube-api-access-pnh4t\") pod \"9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf\" (UID: \"9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf\") " Jul 02 19:26:29.051598 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:29.051585 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf-extensions-socket-volume\") pod \"9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf\" (UID: \"9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf\") " Jul 02 19:26:29.051959 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:29.051937 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf" (UID: "9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:26:29.053572 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:29.053551 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf-kube-api-access-pnh4t" (OuterVolumeSpecName: "kube-api-access-pnh4t") pod "9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf" (UID: "9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf"). InnerVolumeSpecName "kube-api-access-pnh4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:26:29.152731 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:29.152705 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf-extensions-socket-volume\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:26:29.152731 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:29.152728 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnh4t\" (UniqueName: \"kubernetes.io/projected/9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf-kube-api-access-pnh4t\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:26:29.304851 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:29.304776 2578 generic.go:356] "Generic (PLEG): container finished" podID="9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf" containerID="0878a472ce7db50da307f37da6373c0a0a0ffa51e4c96a973d5bfc136c535f07" exitCode=0 Jul 02 19:26:29.304851 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:29.304818 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" Jul 02 19:26:29.305031 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:29.304860 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" event={"ID":"9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf","Type":"ContainerDied","Data":"0878a472ce7db50da307f37da6373c0a0a0ffa51e4c96a973d5bfc136c535f07"} Jul 02 19:26:29.305031 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:29.304910 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk" event={"ID":"9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf","Type":"ContainerDied","Data":"58e260084486367407729afb4641797e42865982520f04281ca41546fb1b8fbe"} Jul 02 19:26:29.305031 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:29.304932 2578 scope.go:117] "RemoveContainer" containerID="0878a472ce7db50da307f37da6373c0a0a0ffa51e4c96a973d5bfc136c535f07" Jul 02 19:26:29.314735 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:29.314720 2578 scope.go:117] "RemoveContainer" containerID="0878a472ce7db50da307f37da6373c0a0a0ffa51e4c96a973d5bfc136c535f07" Jul 02 19:26:29.314990 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:26:29.314958 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0878a472ce7db50da307f37da6373c0a0a0ffa51e4c96a973d5bfc136c535f07\": container with ID starting with 0878a472ce7db50da307f37da6373c0a0a0ffa51e4c96a973d5bfc136c535f07 not found: ID does not exist" containerID="0878a472ce7db50da307f37da6373c0a0a0ffa51e4c96a973d5bfc136c535f07" Jul 02 19:26:29.315041 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:29.314996 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0878a472ce7db50da307f37da6373c0a0a0ffa51e4c96a973d5bfc136c535f07"} err="failed to get container status \"0878a472ce7db50da307f37da6373c0a0a0ffa51e4c96a973d5bfc136c535f07\": rpc error: code = NotFound desc = could not find container \"0878a472ce7db50da307f37da6373c0a0a0ffa51e4c96a973d5bfc136c535f07\": container with ID starting with 0878a472ce7db50da307f37da6373c0a0a0ffa51e4c96a973d5bfc136c535f07 not found: ID does not exist" Jul 02 19:26:29.328473 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:29.328440 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk"] Jul 02 19:26:29.333183 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:29.333162 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mwmdk"] Jul 02 19:26:29.728087 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:26:29.728051 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf" path="/var/lib/kubelet/pods/9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf/volumes" Jul 02 19:27:27.422210 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.422136 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gcbgd"] Jul 02 19:27:27.422719 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.422557 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2be306f0-83a0-4861-be80-5776d297e0d7" containerName="authorino" Jul 02 19:27:27.422719 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.422570 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2be306f0-83a0-4861-be80-5776d297e0d7" containerName="authorino" Jul 02 19:27:27.422719 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.422584 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf" containerName="manager" Jul 02 19:27:27.422719 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.422590 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf" containerName="manager" Jul 02 19:27:27.422719 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.422653 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a457d1c-ce3a-48d3-8d7b-db4eb34d75cf" containerName="manager" Jul 02 19:27:27.422719 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.422664 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2be306f0-83a0-4861-be80-5776d297e0d7" containerName="authorino" Jul 02 19:27:27.425761 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.425743 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gcbgd" Jul 02 19:27:27.428243 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.428221 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-m9ktd\"" Jul 02 19:27:27.437029 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.437001 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gcbgd"] Jul 02 19:27:27.453352 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.453326 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b22ad585-c1dd-4194-a7a0-f677e1e903cf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gcbgd\" (UID: \"b22ad585-c1dd-4194-a7a0-f677e1e903cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gcbgd" Jul 02 19:27:27.453482 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.453385 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w565b\" (UniqueName: \"kubernetes.io/projected/b22ad585-c1dd-4194-a7a0-f677e1e903cf-kube-api-access-w565b\") pod \"kuadrant-operator-controller-manager-86878c769f-gcbgd\" (UID: \"b22ad585-c1dd-4194-a7a0-f677e1e903cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gcbgd" Jul 02 19:27:27.554779 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.554752 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w565b\" (UniqueName: \"kubernetes.io/projected/b22ad585-c1dd-4194-a7a0-f677e1e903cf-kube-api-access-w565b\") pod \"kuadrant-operator-controller-manager-86878c769f-gcbgd\" (UID: \"b22ad585-c1dd-4194-a7a0-f677e1e903cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gcbgd" Jul 02 19:27:27.554907 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.554805 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b22ad585-c1dd-4194-a7a0-f677e1e903cf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gcbgd\" (UID: \"b22ad585-c1dd-4194-a7a0-f677e1e903cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gcbgd" Jul 02 19:27:27.555130 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.555115 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b22ad585-c1dd-4194-a7a0-f677e1e903cf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gcbgd\" (UID: \"b22ad585-c1dd-4194-a7a0-f677e1e903cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gcbgd" Jul 02 19:27:27.563078 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.563048 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w565b\" (UniqueName: \"kubernetes.io/projected/b22ad585-c1dd-4194-a7a0-f677e1e903cf-kube-api-access-w565b\") pod \"kuadrant-operator-controller-manager-86878c769f-gcbgd\" (UID: \"b22ad585-c1dd-4194-a7a0-f677e1e903cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gcbgd" Jul 02 19:27:27.736474 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.736380 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gcbgd" Jul 02 19:27:27.877571 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:27.877549 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gcbgd"] Jul 02 19:27:27.879499 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:27:27.879471 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb22ad585_c1dd_4194_a7a0_f677e1e903cf.slice/crio-fdbe6f7883925cb34a908d07c8388037d7ab81dd5f737505b43bddb2ed7e39b7 WatchSource:0}: Error finding container fdbe6f7883925cb34a908d07c8388037d7ab81dd5f737505b43bddb2ed7e39b7: Status 404 returned error can't find the container with id fdbe6f7883925cb34a908d07c8388037d7ab81dd5f737505b43bddb2ed7e39b7 Jul 02 19:27:28.615075 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:28.615039 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gcbgd" event={"ID":"b22ad585-c1dd-4194-a7a0-f677e1e903cf","Type":"ContainerStarted","Data":"a2798325586a767ce2da6bd4f57be800551e9a927b7be3ef161607306b8d92ef"} Jul 02 19:27:28.615522 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:28.615088 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gcbgd" event={"ID":"b22ad585-c1dd-4194-a7a0-f677e1e903cf","Type":"ContainerStarted","Data":"fdbe6f7883925cb34a908d07c8388037d7ab81dd5f737505b43bddb2ed7e39b7"} Jul 02 19:27:28.615522 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:28.615143 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gcbgd" Jul 02 19:27:28.652258 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:28.652197 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gcbgd" podStartSLOduration=1.652180085 podStartE2EDuration="1.652180085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:27:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:27:28.649016879 +0000 UTC m=+1191.451832329" watchObservedRunningTime="2026-07-02 19:27:28.652180085 +0000 UTC m=+1191.454995536" Jul 02 19:27:39.623321 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:27:39.623287 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gcbgd" Jul 02 19:33:41.688581 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.688556 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2"] Jul 02 19:33:41.692228 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.692213 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.695088 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.695071 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-3747bc9f-openshift-default-dockercfg-z4v4d\"" Jul 02 19:33:41.717305 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.717278 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2"] Jul 02 19:33:41.791205 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.791173 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-istio-data\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.791415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.791243 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89cc982f-85ad-4728-b17d-2683369e04f8-istio-podinfo\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.791415 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.791364 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89cc982f-85ad-4728-b17d-2683369e04f8-istio-token\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.791568 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.791464 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-credential-socket\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.791568 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.791496 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-workload-certs\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.791568 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.791547 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-workload-socket\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.791733 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.791570 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nttjf\" (UniqueName: \"kubernetes.io/projected/89cc982f-85ad-4728-b17d-2683369e04f8-kube-api-access-nttjf\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.791733 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.791597 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-istio-envoy\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.791733 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.791654 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89cc982f-85ad-4728-b17d-2683369e04f8-istiod-ca-cert\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.892467 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.892442 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89cc982f-85ad-4728-b17d-2683369e04f8-istio-podinfo\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.892623 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.892471 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89cc982f-85ad-4728-b17d-2683369e04f8-istio-token\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.892623 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.892503 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-credential-socket\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.892623 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.892530 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-workload-certs\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.892623 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.892576 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-workload-socket\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.892623 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.892599 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nttjf\" (UniqueName: \"kubernetes.io/projected/89cc982f-85ad-4728-b17d-2683369e04f8-kube-api-access-nttjf\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.892910 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.892628 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-istio-envoy\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.892910 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.892662 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89cc982f-85ad-4728-b17d-2683369e04f8-istiod-ca-cert\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.892910 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.892737 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-istio-data\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.893040 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.892905 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-credential-socket\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.893040 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.892998 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-workload-certs\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.893130 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.893073 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-workload-socket\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.893170 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.893138 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-istio-data\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.893613 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.893590 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89cc982f-85ad-4728-b17d-2683369e04f8-istiod-ca-cert\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.894955 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.894936 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-istio-envoy\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.895191 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.895161 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89cc982f-85ad-4728-b17d-2683369e04f8-istio-podinfo\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.900993 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.900972 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89cc982f-85ad-4728-b17d-2683369e04f8-istio-token\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:41.901176 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:41.901156 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nttjf\" (UniqueName: \"kubernetes.io/projected/89cc982f-85ad-4728-b17d-2683369e04f8-kube-api-access-nttjf\") pod \"e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:42.002281 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:42.002221 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:42.144583 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:42.144557 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2"] Jul 02 19:33:42.146231 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:33:42.146208 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89cc982f_85ad_4728_b17d_2683369e04f8.slice/crio-daaf428ef20700d382a6df5c17504f54472246233a013416762a636aa4d10b82 WatchSource:0}: Error finding container daaf428ef20700d382a6df5c17504f54472246233a013416762a636aa4d10b82: Status 404 returned error can't find the container with id daaf428ef20700d382a6df5c17504f54472246233a013416762a636aa4d10b82 Jul 02 19:33:42.148332 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:42.148292 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:33:42.148430 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:42.148379 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:33:42.148478 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:42.148439 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:33:42.600647 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:42.600603 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" event={"ID":"89cc982f-85ad-4728-b17d-2683369e04f8","Type":"ContainerStarted","Data":"d8922093d72714b9f525846df13a2308631c7aa2552efacc095acf01cf0cc0e8"} Jul 02 19:33:42.600837 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:42.600661 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" event={"ID":"89cc982f-85ad-4728-b17d-2683369e04f8","Type":"ContainerStarted","Data":"daaf428ef20700d382a6df5c17504f54472246233a013416762a636aa4d10b82"} Jul 02 19:33:42.622888 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:42.622839 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" podStartSLOduration=1.6228234540000002 podStartE2EDuration="1.622823454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:33:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:33:42.62005158 +0000 UTC m=+1565.422867030" watchObservedRunningTime="2026-07-02 19:33:42.622823454 +0000 UTC m=+1565.425638905" Jul 02 19:33:43.002602 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:43.002537 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:43.007553 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:43.007526 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:43.605745 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:43.605718 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:43.606684 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:43.606665 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:48.193614 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:48.193584 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2"] Jul 02 19:33:48.194046 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:48.193760 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" podUID="89cc982f-85ad-4728-b17d-2683369e04f8" containerName="istio-proxy" containerID="cri-o://d8922093d72714b9f525846df13a2308631c7aa2552efacc095acf01cf0cc0e8" gracePeriod=30 Jul 02 19:33:53.950814 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:53.950750 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:54.108700 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.108669 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nttjf\" (UniqueName: \"kubernetes.io/projected/89cc982f-85ad-4728-b17d-2683369e04f8-kube-api-access-nttjf\") pod \"89cc982f-85ad-4728-b17d-2683369e04f8\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " Jul 02 19:33:54.108700 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.108702 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-workload-certs\") pod \"89cc982f-85ad-4728-b17d-2683369e04f8\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " Jul 02 19:33:54.108936 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.108723 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-istio-data\") pod \"89cc982f-85ad-4728-b17d-2683369e04f8\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " Jul 02 19:33:54.108936 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.108757 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-credential-socket\") pod \"89cc982f-85ad-4728-b17d-2683369e04f8\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " Jul 02 19:33:54.108936 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.108786 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-istio-envoy\") pod \"89cc982f-85ad-4728-b17d-2683369e04f8\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " Jul 02 19:33:54.108936 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.108820 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89cc982f-85ad-4728-b17d-2683369e04f8-istiod-ca-cert\") pod \"89cc982f-85ad-4728-b17d-2683369e04f8\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " Jul 02 19:33:54.108936 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.108850 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89cc982f-85ad-4728-b17d-2683369e04f8-istio-podinfo\") pod \"89cc982f-85ad-4728-b17d-2683369e04f8\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " Jul 02 19:33:54.108936 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.108926 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89cc982f-85ad-4728-b17d-2683369e04f8-istio-token\") pod \"89cc982f-85ad-4728-b17d-2683369e04f8\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " Jul 02 19:33:54.109272 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.108947 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-workload-socket\") pod \"89cc982f-85ad-4728-b17d-2683369e04f8\" (UID: \"89cc982f-85ad-4728-b17d-2683369e04f8\") " Jul 02 19:33:54.109272 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.108988 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "89cc982f-85ad-4728-b17d-2683369e04f8" (UID: "89cc982f-85ad-4728-b17d-2683369e04f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:33:54.109272 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.109154 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "89cc982f-85ad-4728-b17d-2683369e04f8" (UID: "89cc982f-85ad-4728-b17d-2683369e04f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:33:54.109272 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.109225 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-workload-certs\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:33:54.109272 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.109244 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-credential-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:33:54.109272 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.109260 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89cc982f-85ad-4728-b17d-2683369e04f8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "89cc982f-85ad-4728-b17d-2683369e04f8" (UID: "89cc982f-85ad-4728-b17d-2683369e04f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:33:54.109614 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.109350 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "89cc982f-85ad-4728-b17d-2683369e04f8" (UID: "89cc982f-85ad-4728-b17d-2683369e04f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:33:54.109614 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.109604 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-istio-data" (OuterVolumeSpecName: "istio-data") pod "89cc982f-85ad-4728-b17d-2683369e04f8" (UID: "89cc982f-85ad-4728-b17d-2683369e04f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:33:54.111100 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.111068 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "89cc982f-85ad-4728-b17d-2683369e04f8" (UID: "89cc982f-85ad-4728-b17d-2683369e04f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:33:54.111216 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.111132 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89cc982f-85ad-4728-b17d-2683369e04f8-kube-api-access-nttjf" (OuterVolumeSpecName: "kube-api-access-nttjf") pod "89cc982f-85ad-4728-b17d-2683369e04f8" (UID: "89cc982f-85ad-4728-b17d-2683369e04f8"). InnerVolumeSpecName "kube-api-access-nttjf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:33:54.111216 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.111157 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89cc982f-85ad-4728-b17d-2683369e04f8-istio-token" (OuterVolumeSpecName: "istio-token") pod "89cc982f-85ad-4728-b17d-2683369e04f8" (UID: "89cc982f-85ad-4728-b17d-2683369e04f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:33:54.111216 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.111199 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/89cc982f-85ad-4728-b17d-2683369e04f8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "89cc982f-85ad-4728-b17d-2683369e04f8" (UID: "89cc982f-85ad-4728-b17d-2683369e04f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:33:54.209852 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.209795 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89cc982f-85ad-4728-b17d-2683369e04f8-istio-token\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:33:54.209852 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.209815 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-workload-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:33:54.209852 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.209825 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nttjf\" (UniqueName: \"kubernetes.io/projected/89cc982f-85ad-4728-b17d-2683369e04f8-kube-api-access-nttjf\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:33:54.209852 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.209834 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-istio-data\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:33:54.209852 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.209842 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89cc982f-85ad-4728-b17d-2683369e04f8-istio-envoy\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:33:54.209852 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.209850 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89cc982f-85ad-4728-b17d-2683369e04f8-istiod-ca-cert\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:33:54.209852 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.209858 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89cc982f-85ad-4728-b17d-2683369e04f8-istio-podinfo\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:33:54.501825 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.501748 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh"] Jul 02 19:33:54.502324 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.502300 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89cc982f-85ad-4728-b17d-2683369e04f8" containerName="istio-proxy" Jul 02 19:33:54.502324 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.502324 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="89cc982f-85ad-4728-b17d-2683369e04f8" containerName="istio-proxy" Jul 02 19:33:54.502508 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.502471 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="89cc982f-85ad-4728-b17d-2683369e04f8" containerName="istio-proxy" Jul 02 19:33:54.506225 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.506201 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.508518 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.508498 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f765905c-openshift-default-dockercfg-knlkx\"" Jul 02 19:33:54.519640 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.519619 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh"] Jul 02 19:33:54.613160 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.613131 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-envoy\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.613160 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.613163 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istiod-ca-cert\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.613361 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.613187 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgjh5\" (UniqueName: \"kubernetes.io/projected/9799f92c-e0e6-4bd0-a957-03aa0b92c483-kube-api-access-vgjh5\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.613361 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.613256 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-workload-socket\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.613361 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.613283 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-data\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.613361 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.613315 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-credential-socket\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.613516 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.613384 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-podinfo\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.613516 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.613436 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-token\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.613516 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.613462 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-workload-certs\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.670414 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.670356 2578 generic.go:356] "Generic (PLEG): container finished" podID="89cc982f-85ad-4728-b17d-2683369e04f8" containerID="d8922093d72714b9f525846df13a2308631c7aa2552efacc095acf01cf0cc0e8" exitCode=0 Jul 02 19:33:54.670579 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.670421 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" Jul 02 19:33:54.670579 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.670445 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" event={"ID":"89cc982f-85ad-4728-b17d-2683369e04f8","Type":"ContainerDied","Data":"d8922093d72714b9f525846df13a2308631c7aa2552efacc095acf01cf0cc0e8"} Jul 02 19:33:54.670579 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.670494 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2" event={"ID":"89cc982f-85ad-4728-b17d-2683369e04f8","Type":"ContainerDied","Data":"daaf428ef20700d382a6df5c17504f54472246233a013416762a636aa4d10b82"} Jul 02 19:33:54.670579 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.670517 2578 scope.go:117] "RemoveContainer" containerID="d8922093d72714b9f525846df13a2308631c7aa2552efacc095acf01cf0cc0e8" Jul 02 19:33:54.679586 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.679572 2578 scope.go:117] "RemoveContainer" containerID="d8922093d72714b9f525846df13a2308631c7aa2552efacc095acf01cf0cc0e8" Jul 02 19:33:54.679825 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:33:54.679807 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8922093d72714b9f525846df13a2308631c7aa2552efacc095acf01cf0cc0e8\": container with ID starting with d8922093d72714b9f525846df13a2308631c7aa2552efacc095acf01cf0cc0e8 not found: ID does not exist" containerID="d8922093d72714b9f525846df13a2308631c7aa2552efacc095acf01cf0cc0e8" Jul 02 19:33:54.679889 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.679837 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8922093d72714b9f525846df13a2308631c7aa2552efacc095acf01cf0cc0e8"} err="failed to get container status \"d8922093d72714b9f525846df13a2308631c7aa2552efacc095acf01cf0cc0e8\": rpc error: code = NotFound desc = could not find container \"d8922093d72714b9f525846df13a2308631c7aa2552efacc095acf01cf0cc0e8\": container with ID starting with d8922093d72714b9f525846df13a2308631c7aa2552efacc095acf01cf0cc0e8 not found: ID does not exist" Jul 02 19:33:54.694026 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.694007 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2"] Jul 02 19:33:54.698047 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.698027 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-3747bc9f-openshift-default-585b9bb86f-qx6r2"] Jul 02 19:33:54.714831 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.714812 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-envoy\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.714926 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.714837 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istiod-ca-cert\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.714926 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.714866 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgjh5\" (UniqueName: \"kubernetes.io/projected/9799f92c-e0e6-4bd0-a957-03aa0b92c483-kube-api-access-vgjh5\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.714926 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.714891 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-workload-socket\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.715045 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.714999 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-data\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.715112 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.715055 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-credential-socket\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.715112 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.715099 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-podinfo\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.715230 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.715144 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-token\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.715230 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.715187 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-workload-certs\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.715332 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.715265 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-workload-socket\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.715332 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.715311 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-data\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.715497 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.715475 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-credential-socket\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.715570 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.715526 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-workload-certs\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.715766 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.715746 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istiod-ca-cert\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.717058 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.717032 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-envoy\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.717501 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.717483 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-podinfo\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.723696 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.723671 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-token\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.723776 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.723701 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgjh5\" (UniqueName: \"kubernetes.io/projected/9799f92c-e0e6-4bd0-a957-03aa0b92c483-kube-api-access-vgjh5\") pod \"e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.820103 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.820077 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:54.962674 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.962648 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh"] Jul 02 19:33:54.963568 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:33:54.963533 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9799f92c_e0e6_4bd0_a957_03aa0b92c483.slice/crio-7c0d6df4adb6ecf1974762534797c316402242e5859ba197c88074164523b4a2 WatchSource:0}: Error finding container 7c0d6df4adb6ecf1974762534797c316402242e5859ba197c88074164523b4a2: Status 404 returned error can't find the container with id 7c0d6df4adb6ecf1974762534797c316402242e5859ba197c88074164523b4a2 Jul 02 19:33:54.966022 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.965989 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:33:54.966111 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.966052 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:33:54.966111 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:54.966081 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:33:55.678073 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:55.678033 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" event={"ID":"9799f92c-e0e6-4bd0-a957-03aa0b92c483","Type":"ContainerStarted","Data":"33972f1a43707b1ad41afe9d0c7cebb4be269acdddf022639bbc36806637f650"} Jul 02 19:33:55.678252 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:55.678093 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" event={"ID":"9799f92c-e0e6-4bd0-a957-03aa0b92c483","Type":"ContainerStarted","Data":"7c0d6df4adb6ecf1974762534797c316402242e5859ba197c88074164523b4a2"} Jul 02 19:33:55.709162 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:55.709118 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" podStartSLOduration=1.709101631 podStartE2EDuration="1.709101631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:33:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:33:55.707593099 +0000 UTC m=+1578.510408550" watchObservedRunningTime="2026-07-02 19:33:55.709101631 +0000 UTC m=+1578.511917146" Jul 02 19:33:55.729155 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:55.729129 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89cc982f-85ad-4728-b17d-2683369e04f8" path="/var/lib/kubelet/pods/89cc982f-85ad-4728-b17d-2683369e04f8/volumes" Jul 02 19:33:55.820753 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:55.820724 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:56.824956 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:56.824929 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:57.689574 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:57.689541 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:33:57.690827 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:33:57.690806 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:34:06.740632 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:06.740598 2578 generic.go:356] "Generic (PLEG): container finished" podID="9799f92c-e0e6-4bd0-a957-03aa0b92c483" containerID="33972f1a43707b1ad41afe9d0c7cebb4be269acdddf022639bbc36806637f650" exitCode=0 Jul 02 19:34:06.741079 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:06.740675 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" event={"ID":"9799f92c-e0e6-4bd0-a957-03aa0b92c483","Type":"ContainerDied","Data":"33972f1a43707b1ad41afe9d0c7cebb4be269acdddf022639bbc36806637f650"} Jul 02 19:34:06.741162 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:06.741147 2578 scope.go:117] "RemoveContainer" containerID="33972f1a43707b1ad41afe9d0c7cebb4be269acdddf022639bbc36806637f650" Jul 02 19:34:06.742170 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:06.742141 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:34:06.742234 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:06.742210 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:34:06.742271 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:06.742246 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:34:07.198474 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:07.198439 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh"] Jul 02 19:34:07.747242 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:07.747204 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" event={"ID":"9799f92c-e0e6-4bd0-a957-03aa0b92c483","Type":"ContainerStarted","Data":"edfec4ca041abcfc1ae946e6551c67dd9bd6a3b00acef4f60ed0fbdce2cb1c30"} Jul 02 19:34:07.747732 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:07.747268 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" podUID="9799f92c-e0e6-4bd0-a957-03aa0b92c483" containerName="istio-proxy" containerID="cri-o://edfec4ca041abcfc1ae946e6551c67dd9bd6a3b00acef4f60ed0fbdce2cb1c30" gracePeriod=30 Jul 02 19:34:12.690120 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:12.690088 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:34:12.991659 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:12.991632 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:34:13.072784 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.072758 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgjh5\" (UniqueName: \"kubernetes.io/projected/9799f92c-e0e6-4bd0-a957-03aa0b92c483-kube-api-access-vgjh5\") pod \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " Jul 02 19:34:13.072951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.072790 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istiod-ca-cert\") pod \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " Jul 02 19:34:13.072951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.072822 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-workload-socket\") pod \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " Jul 02 19:34:13.072951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.072857 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-token\") pod \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " Jul 02 19:34:13.072951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.072897 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-credential-socket\") pod \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " Jul 02 19:34:13.072951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.072928 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-data\") pod \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " Jul 02 19:34:13.073235 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.072970 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-envoy\") pod \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " Jul 02 19:34:13.073235 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.072994 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-podinfo\") pod \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " Jul 02 19:34:13.073235 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.073019 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-workload-certs\") pod \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\" (UID: \"9799f92c-e0e6-4bd0-a957-03aa0b92c483\") " Jul 02 19:34:13.073235 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.073133 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9799f92c-e0e6-4bd0-a957-03aa0b92c483" (UID: "9799f92c-e0e6-4bd0-a957-03aa0b92c483"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:34:13.073235 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.073159 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9799f92c-e0e6-4bd0-a957-03aa0b92c483" (UID: "9799f92c-e0e6-4bd0-a957-03aa0b92c483"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:34:13.073478 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.073325 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istiod-ca-cert\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:13.073478 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.073338 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9799f92c-e0e6-4bd0-a957-03aa0b92c483" (UID: "9799f92c-e0e6-4bd0-a957-03aa0b92c483"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:34:13.073478 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.073353 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9799f92c-e0e6-4bd0-a957-03aa0b92c483" (UID: "9799f92c-e0e6-4bd0-a957-03aa0b92c483"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:34:13.073478 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.073344 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-workload-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:13.073850 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.073827 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-data" (OuterVolumeSpecName: "istio-data") pod "9799f92c-e0e6-4bd0-a957-03aa0b92c483" (UID: "9799f92c-e0e6-4bd0-a957-03aa0b92c483"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:34:13.075166 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.075138 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9799f92c-e0e6-4bd0-a957-03aa0b92c483-kube-api-access-vgjh5" (OuterVolumeSpecName: "kube-api-access-vgjh5") pod "9799f92c-e0e6-4bd0-a957-03aa0b92c483" (UID: "9799f92c-e0e6-4bd0-a957-03aa0b92c483"). InnerVolumeSpecName "kube-api-access-vgjh5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:34:13.075166 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.075161 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9799f92c-e0e6-4bd0-a957-03aa0b92c483" (UID: "9799f92c-e0e6-4bd0-a957-03aa0b92c483"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:34:13.075575 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.075548 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-token" (OuterVolumeSpecName: "istio-token") pod "9799f92c-e0e6-4bd0-a957-03aa0b92c483" (UID: "9799f92c-e0e6-4bd0-a957-03aa0b92c483"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:34:13.075575 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.075564 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9799f92c-e0e6-4bd0-a957-03aa0b92c483" (UID: "9799f92c-e0e6-4bd0-a957-03aa0b92c483"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:34:13.174281 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.174254 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-credential-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:13.174281 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.174276 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-data\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:13.174281 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.174286 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-envoy\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:13.174500 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.174294 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-podinfo\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:13.174500 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.174302 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9799f92c-e0e6-4bd0-a957-03aa0b92c483-workload-certs\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:13.174500 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.174311 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgjh5\" (UniqueName: \"kubernetes.io/projected/9799f92c-e0e6-4bd0-a957-03aa0b92c483-kube-api-access-vgjh5\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:13.174500 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.174321 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9799f92c-e0e6-4bd0-a957-03aa0b92c483-istio-token\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:13.370973 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.370941 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz"] Jul 02 19:34:13.371378 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.371361 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9799f92c-e0e6-4bd0-a957-03aa0b92c483" containerName="istio-proxy" Jul 02 19:34:13.371500 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.371381 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9799f92c-e0e6-4bd0-a957-03aa0b92c483" containerName="istio-proxy" Jul 02 19:34:13.371500 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.371411 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9799f92c-e0e6-4bd0-a957-03aa0b92c483" containerName="istio-proxy" Jul 02 19:34:13.371500 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.371420 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9799f92c-e0e6-4bd0-a957-03aa0b92c483" containerName="istio-proxy" Jul 02 19:34:13.371657 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.371503 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9799f92c-e0e6-4bd0-a957-03aa0b92c483" containerName="istio-proxy" Jul 02 19:34:13.371657 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.371519 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9799f92c-e0e6-4bd0-a957-03aa0b92c483" containerName="istio-proxy" Jul 02 19:34:13.374916 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.374890 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.377651 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.377629 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-85690430-openshift-default-dockercfg-mf54g\"" Jul 02 19:34:13.391632 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.391611 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz"] Jul 02 19:34:13.477187 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.477151 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-istio-data\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.477371 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.477200 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-istio-envoy\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.477371 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.477242 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/079ca4f1-919b-43b5-b87f-838be33409e9-istio-podinfo\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.477371 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.477298 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-workload-socket\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.477371 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.477329 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-workload-certs\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.477585 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.477414 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/079ca4f1-919b-43b5-b87f-838be33409e9-istio-token\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.477585 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.477442 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/079ca4f1-919b-43b5-b87f-838be33409e9-istiod-ca-cert\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.477585 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.477483 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-credential-socket\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.477585 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.477500 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq4xt\" (UniqueName: \"kubernetes.io/projected/079ca4f1-919b-43b5-b87f-838be33409e9-kube-api-access-nq4xt\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.578738 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.578707 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-workload-socket\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.578738 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.578740 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-workload-certs\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.578939 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.578761 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/079ca4f1-919b-43b5-b87f-838be33409e9-istio-token\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.578939 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.578777 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/079ca4f1-919b-43b5-b87f-838be33409e9-istiod-ca-cert\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.578939 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.578801 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-credential-socket\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.578939 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.578825 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq4xt\" (UniqueName: \"kubernetes.io/projected/079ca4f1-919b-43b5-b87f-838be33409e9-kube-api-access-nq4xt\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.578939 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.578894 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-istio-data\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.579199 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.578922 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-istio-envoy\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.579199 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.578997 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/079ca4f1-919b-43b5-b87f-838be33409e9-istio-podinfo\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.579301 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.579215 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-workload-socket\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.579301 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.579215 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-workload-certs\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.579437 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.579335 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-istio-data\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.579437 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.579414 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-credential-socket\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.579612 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.579590 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/079ca4f1-919b-43b5-b87f-838be33409e9-istiod-ca-cert\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.581353 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.581334 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-istio-envoy\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.581496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.581480 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/079ca4f1-919b-43b5-b87f-838be33409e9-istio-podinfo\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.586653 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.586629 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/079ca4f1-919b-43b5-b87f-838be33409e9-istio-token\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.586873 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.586857 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq4xt\" (UniqueName: \"kubernetes.io/projected/079ca4f1-919b-43b5-b87f-838be33409e9-kube-api-access-nq4xt\") pod \"e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.685703 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.685648 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:13.783342 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.783308 2578 generic.go:356] "Generic (PLEG): container finished" podID="9799f92c-e0e6-4bd0-a957-03aa0b92c483" containerID="edfec4ca041abcfc1ae946e6551c67dd9bd6a3b00acef4f60ed0fbdce2cb1c30" exitCode=0 Jul 02 19:34:13.783763 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.783410 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" event={"ID":"9799f92c-e0e6-4bd0-a957-03aa0b92c483","Type":"ContainerDied","Data":"edfec4ca041abcfc1ae946e6551c67dd9bd6a3b00acef4f60ed0fbdce2cb1c30"} Jul 02 19:34:13.783763 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.783474 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" event={"ID":"9799f92c-e0e6-4bd0-a957-03aa0b92c483","Type":"ContainerDied","Data":"7c0d6df4adb6ecf1974762534797c316402242e5859ba197c88074164523b4a2"} Jul 02 19:34:13.783763 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.783500 2578 scope.go:117] "RemoveContainer" containerID="edfec4ca041abcfc1ae946e6551c67dd9bd6a3b00acef4f60ed0fbdce2cb1c30" Jul 02 19:34:13.783763 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.783512 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh" Jul 02 19:34:13.795649 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.795634 2578 scope.go:117] "RemoveContainer" containerID="33972f1a43707b1ad41afe9d0c7cebb4be269acdddf022639bbc36806637f650" Jul 02 19:34:13.802813 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.802787 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh"] Jul 02 19:34:13.808381 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.808361 2578 scope.go:117] "RemoveContainer" containerID="edfec4ca041abcfc1ae946e6551c67dd9bd6a3b00acef4f60ed0fbdce2cb1c30" Jul 02 19:34:13.808778 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:34:13.808747 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edfec4ca041abcfc1ae946e6551c67dd9bd6a3b00acef4f60ed0fbdce2cb1c30\": container with ID starting with edfec4ca041abcfc1ae946e6551c67dd9bd6a3b00acef4f60ed0fbdce2cb1c30 not found: ID does not exist" containerID="edfec4ca041abcfc1ae946e6551c67dd9bd6a3b00acef4f60ed0fbdce2cb1c30" Jul 02 19:34:13.808883 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.808788 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edfec4ca041abcfc1ae946e6551c67dd9bd6a3b00acef4f60ed0fbdce2cb1c30"} err="failed to get container status \"edfec4ca041abcfc1ae946e6551c67dd9bd6a3b00acef4f60ed0fbdce2cb1c30\": rpc error: code = NotFound desc = could not find container \"edfec4ca041abcfc1ae946e6551c67dd9bd6a3b00acef4f60ed0fbdce2cb1c30\": container with ID starting with edfec4ca041abcfc1ae946e6551c67dd9bd6a3b00acef4f60ed0fbdce2cb1c30 not found: ID does not exist" Jul 02 19:34:13.808883 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.808815 2578 scope.go:117] "RemoveContainer" containerID="33972f1a43707b1ad41afe9d0c7cebb4be269acdddf022639bbc36806637f650" Jul 02 19:34:13.809091 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.809073 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f765905c-openshift-default-cc85b9c7-bz8hh"] Jul 02 19:34:13.809140 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:34:13.809073 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33972f1a43707b1ad41afe9d0c7cebb4be269acdddf022639bbc36806637f650\": container with ID starting with 33972f1a43707b1ad41afe9d0c7cebb4be269acdddf022639bbc36806637f650 not found: ID does not exist" containerID="33972f1a43707b1ad41afe9d0c7cebb4be269acdddf022639bbc36806637f650" Jul 02 19:34:13.809183 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.809132 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33972f1a43707b1ad41afe9d0c7cebb4be269acdddf022639bbc36806637f650"} err="failed to get container status \"33972f1a43707b1ad41afe9d0c7cebb4be269acdddf022639bbc36806637f650\": rpc error: code = NotFound desc = could not find container \"33972f1a43707b1ad41afe9d0c7cebb4be269acdddf022639bbc36806637f650\": container with ID starting with 33972f1a43707b1ad41afe9d0c7cebb4be269acdddf022639bbc36806637f650 not found: ID does not exist" Jul 02 19:34:13.833069 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.833037 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz"] Jul 02 19:34:13.834182 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:34:13.834155 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod079ca4f1_919b_43b5_b87f_838be33409e9.slice/crio-6190cfcd80233b23375a6218ea0b7c5d96403d49453bbce8405215a733d5b26a WatchSource:0}: Error finding container 6190cfcd80233b23375a6218ea0b7c5d96403d49453bbce8405215a733d5b26a: Status 404 returned error can't find the container with id 6190cfcd80233b23375a6218ea0b7c5d96403d49453bbce8405215a733d5b26a Jul 02 19:34:13.836900 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.836863 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:34:13.837009 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.836946 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:34:13.837009 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:13.836989 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:34:14.791382 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:14.791343 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" event={"ID":"079ca4f1-919b-43b5-b87f-838be33409e9","Type":"ContainerStarted","Data":"a90472f44f41d0fd9a90e642393bde877459591aafdf1595f504ad9a2c6f5f81"} Jul 02 19:34:14.791817 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:14.791429 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" event={"ID":"079ca4f1-919b-43b5-b87f-838be33409e9","Type":"ContainerStarted","Data":"6190cfcd80233b23375a6218ea0b7c5d96403d49453bbce8405215a733d5b26a"} Jul 02 19:34:14.817003 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:14.816958 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" podStartSLOduration=1.816943333 podStartE2EDuration="1.816943333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:34:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:34:14.815010637 +0000 UTC m=+1597.617826088" watchObservedRunningTime="2026-07-02 19:34:14.816943333 +0000 UTC m=+1597.619758784" Jul 02 19:34:15.685810 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:15.685775 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:15.691216 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:15.691194 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:15.727285 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:15.727257 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9799f92c-e0e6-4bd0-a957-03aa0b92c483" path="/var/lib/kubelet/pods/9799f92c-e0e6-4bd0-a957-03aa0b92c483/volumes" Jul 02 19:34:15.797386 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:15.797361 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:15.798359 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:15.798343 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:19.169307 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:19.169272 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz"] Jul 02 19:34:19.169704 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:19.169474 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" podUID="079ca4f1-919b-43b5-b87f-838be33409e9" containerName="istio-proxy" containerID="cri-o://a90472f44f41d0fd9a90e642393bde877459591aafdf1595f504ad9a2c6f5f81" gracePeriod=30 Jul 02 19:34:24.432317 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.432254 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:24.472942 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.472918 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/079ca4f1-919b-43b5-b87f-838be33409e9-istio-podinfo\") pod \"079ca4f1-919b-43b5-b87f-838be33409e9\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " Jul 02 19:34:24.473085 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.472951 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/079ca4f1-919b-43b5-b87f-838be33409e9-istiod-ca-cert\") pod \"079ca4f1-919b-43b5-b87f-838be33409e9\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " Jul 02 19:34:24.473085 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.473009 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-istio-data\") pod \"079ca4f1-919b-43b5-b87f-838be33409e9\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " Jul 02 19:34:24.473085 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.473071 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-workload-socket\") pod \"079ca4f1-919b-43b5-b87f-838be33409e9\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " Jul 02 19:34:24.473282 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.473093 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/079ca4f1-919b-43b5-b87f-838be33409e9-istio-token\") pod \"079ca4f1-919b-43b5-b87f-838be33409e9\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " Jul 02 19:34:24.473282 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.473121 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nq4xt\" (UniqueName: \"kubernetes.io/projected/079ca4f1-919b-43b5-b87f-838be33409e9-kube-api-access-nq4xt\") pod \"079ca4f1-919b-43b5-b87f-838be33409e9\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " Jul 02 19:34:24.473282 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.473155 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-credential-socket\") pod \"079ca4f1-919b-43b5-b87f-838be33409e9\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " Jul 02 19:34:24.473282 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.473184 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-istio-envoy\") pod \"079ca4f1-919b-43b5-b87f-838be33409e9\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " Jul 02 19:34:24.473282 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.473233 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-workload-certs\") pod \"079ca4f1-919b-43b5-b87f-838be33409e9\" (UID: \"079ca4f1-919b-43b5-b87f-838be33409e9\") " Jul 02 19:34:24.473536 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.473322 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-istio-data" (OuterVolumeSpecName: "istio-data") pod "079ca4f1-919b-43b5-b87f-838be33409e9" (UID: "079ca4f1-919b-43b5-b87f-838be33409e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:34:24.473536 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.473431 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "079ca4f1-919b-43b5-b87f-838be33409e9" (UID: "079ca4f1-919b-43b5-b87f-838be33409e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:34:24.473536 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.473487 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/079ca4f1-919b-43b5-b87f-838be33409e9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "079ca4f1-919b-43b5-b87f-838be33409e9" (UID: "079ca4f1-919b-43b5-b87f-838be33409e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:34:24.473536 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.473497 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "079ca4f1-919b-43b5-b87f-838be33409e9" (UID: "079ca4f1-919b-43b5-b87f-838be33409e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:34:24.473753 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.473577 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-workload-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:24.473753 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.473596 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-credential-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:24.473753 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.473610 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/079ca4f1-919b-43b5-b87f-838be33409e9-istiod-ca-cert\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:24.473753 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.473623 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-istio-data\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:24.473753 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.473734 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "079ca4f1-919b-43b5-b87f-838be33409e9" (UID: "079ca4f1-919b-43b5-b87f-838be33409e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:34:24.475341 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.475319 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/079ca4f1-919b-43b5-b87f-838be33409e9-kube-api-access-nq4xt" (OuterVolumeSpecName: "kube-api-access-nq4xt") pod "079ca4f1-919b-43b5-b87f-838be33409e9" (UID: "079ca4f1-919b-43b5-b87f-838be33409e9"). InnerVolumeSpecName "kube-api-access-nq4xt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:34:24.475719 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.475695 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/079ca4f1-919b-43b5-b87f-838be33409e9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "079ca4f1-919b-43b5-b87f-838be33409e9" (UID: "079ca4f1-919b-43b5-b87f-838be33409e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:34:24.475719 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.475703 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/079ca4f1-919b-43b5-b87f-838be33409e9-istio-token" (OuterVolumeSpecName: "istio-token") pod "079ca4f1-919b-43b5-b87f-838be33409e9" (UID: "079ca4f1-919b-43b5-b87f-838be33409e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:34:24.475818 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.475793 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "079ca4f1-919b-43b5-b87f-838be33409e9" (UID: "079ca4f1-919b-43b5-b87f-838be33409e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:34:24.574722 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.574698 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-workload-certs\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:24.574722 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.574719 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/079ca4f1-919b-43b5-b87f-838be33409e9-istio-podinfo\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:24.574863 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.574728 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/079ca4f1-919b-43b5-b87f-838be33409e9-istio-token\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:24.574863 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.574737 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq4xt\" (UniqueName: \"kubernetes.io/projected/079ca4f1-919b-43b5-b87f-838be33409e9-kube-api-access-nq4xt\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:24.574863 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.574746 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/079ca4f1-919b-43b5-b87f-838be33409e9-istio-envoy\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:34:24.847831 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.847796 2578 generic.go:356] "Generic (PLEG): container finished" podID="079ca4f1-919b-43b5-b87f-838be33409e9" containerID="a90472f44f41d0fd9a90e642393bde877459591aafdf1595f504ad9a2c6f5f81" exitCode=0 Jul 02 19:34:24.848015 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.847847 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" Jul 02 19:34:24.848015 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.847874 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" event={"ID":"079ca4f1-919b-43b5-b87f-838be33409e9","Type":"ContainerDied","Data":"a90472f44f41d0fd9a90e642393bde877459591aafdf1595f504ad9a2c6f5f81"} Jul 02 19:34:24.848015 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.847924 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz" event={"ID":"079ca4f1-919b-43b5-b87f-838be33409e9","Type":"ContainerDied","Data":"6190cfcd80233b23375a6218ea0b7c5d96403d49453bbce8405215a733d5b26a"} Jul 02 19:34:24.848015 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.847947 2578 scope.go:117] "RemoveContainer" containerID="a90472f44f41d0fd9a90e642393bde877459591aafdf1595f504ad9a2c6f5f81" Jul 02 19:34:24.857605 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.857590 2578 scope.go:117] "RemoveContainer" containerID="a90472f44f41d0fd9a90e642393bde877459591aafdf1595f504ad9a2c6f5f81" Jul 02 19:34:24.857847 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:34:24.857827 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a90472f44f41d0fd9a90e642393bde877459591aafdf1595f504ad9a2c6f5f81\": container with ID starting with a90472f44f41d0fd9a90e642393bde877459591aafdf1595f504ad9a2c6f5f81 not found: ID does not exist" containerID="a90472f44f41d0fd9a90e642393bde877459591aafdf1595f504ad9a2c6f5f81" Jul 02 19:34:24.857920 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.857859 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a90472f44f41d0fd9a90e642393bde877459591aafdf1595f504ad9a2c6f5f81"} err="failed to get container status \"a90472f44f41d0fd9a90e642393bde877459591aafdf1595f504ad9a2c6f5f81\": rpc error: code = NotFound desc = could not find container \"a90472f44f41d0fd9a90e642393bde877459591aafdf1595f504ad9a2c6f5f81\": container with ID starting with a90472f44f41d0fd9a90e642393bde877459591aafdf1595f504ad9a2c6f5f81 not found: ID does not exist" Jul 02 19:34:24.869230 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.869198 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz"] Jul 02 19:34:24.877958 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:24.877931 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-85690430-openshift-default-b96c97ddc-gq6bz"] Jul 02 19:34:25.728213 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:34:25.728184 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="079ca4f1-919b-43b5-b87f-838be33409e9" path="/var/lib/kubelet/pods/079ca4f1-919b-43b5-b87f-838be33409e9/volumes" Jul 02 19:36:02.843390 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:02.843301 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7"] Jul 02 19:36:02.843976 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:02.843860 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="079ca4f1-919b-43b5-b87f-838be33409e9" containerName="istio-proxy" Jul 02 19:36:02.843976 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:02.843881 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="079ca4f1-919b-43b5-b87f-838be33409e9" containerName="istio-proxy" Jul 02 19:36:02.844096 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:02.843989 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="079ca4f1-919b-43b5-b87f-838be33409e9" containerName="istio-proxy" Jul 02 19:36:02.847001 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:02.846982 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:02.849692 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:02.849671 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2cd582b7-openshift-default-dockercfg-w4fv8\"" Jul 02 19:36:02.874682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:02.874650 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7"] Jul 02 19:36:02.915701 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:02.915666 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-workload-socket\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:02.915701 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:02.915702 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-workload-certs\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:02.915894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:02.915725 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-envoy\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:02.915894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:02.915812 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-podinfo\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:02.915894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:02.915844 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-credential-socket\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:02.915894 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:02.915875 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-data\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:02.916034 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:02.915914 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrnt4\" (UniqueName: \"kubernetes.io/projected/a7f638a6-6cb9-4e31-86d3-bf284b5940da-kube-api-access-mrnt4\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:02.916034 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:02.915950 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-token\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:02.916034 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:02.915972 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istiod-ca-cert\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.017477 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.017437 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-workload-socket\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.017477 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.017475 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-workload-certs\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.017744 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.017503 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-envoy\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.017744 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.017521 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-podinfo\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.017744 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.017539 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-credential-socket\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.017744 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.017563 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-data\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.017744 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.017598 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrnt4\" (UniqueName: \"kubernetes.io/projected/a7f638a6-6cb9-4e31-86d3-bf284b5940da-kube-api-access-mrnt4\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.017744 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.017621 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-token\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.017744 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.017648 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istiod-ca-cert\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.018107 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.017955 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-data\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.018107 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.017928 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-workload-socket\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.018107 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.017987 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-credential-socket\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.018107 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.018040 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-workload-certs\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.018431 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.018381 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istiod-ca-cert\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.020000 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.019970 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-envoy\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.020090 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.020044 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-podinfo\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.025956 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.025932 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-token\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.026438 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.026392 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrnt4\" (UniqueName: \"kubernetes.io/projected/a7f638a6-6cb9-4e31-86d3-bf284b5940da-kube-api-access-mrnt4\") pod \"e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.158102 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.158001 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:03.302530 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.302506 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7"] Jul 02 19:36:03.304383 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:36:03.304346 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7f638a6_6cb9_4e31_86d3_bf284b5940da.slice/crio-849bc38f31c4593e346c60b07623117b54ac6901f64242e18df0b53113689c88 WatchSource:0}: Error finding container 849bc38f31c4593e346c60b07623117b54ac6901f64242e18df0b53113689c88: Status 404 returned error can't find the container with id 849bc38f31c4593e346c60b07623117b54ac6901f64242e18df0b53113689c88 Jul 02 19:36:03.306718 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.306678 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:36:03.306804 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.306767 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:36:03.306847 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.306809 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:36:03.369067 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:03.369041 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" event={"ID":"a7f638a6-6cb9-4e31-86d3-bf284b5940da","Type":"ContainerStarted","Data":"849bc38f31c4593e346c60b07623117b54ac6901f64242e18df0b53113689c88"} Jul 02 19:36:04.376163 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:04.376128 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" event={"ID":"a7f638a6-6cb9-4e31-86d3-bf284b5940da","Type":"ContainerStarted","Data":"bb3bb5acb86024306c440b1b8370735b7f28aee15fcbf9f6cc218a52f686a292"} Jul 02 19:36:04.403831 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:04.403780 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" podStartSLOduration=2.403763802 podStartE2EDuration="2.403763802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:36:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:36:04.401300068 +0000 UTC m=+1707.204115523" watchObservedRunningTime="2026-07-02 19:36:04.403763802 +0000 UTC m=+1707.206579252" Jul 02 19:36:05.158568 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:05.158530 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:05.163466 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:05.163439 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:05.381673 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:05.381647 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:05.382712 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:05.382694 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:19.460326 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:19.460291 2578 generic.go:356] "Generic (PLEG): container finished" podID="a7f638a6-6cb9-4e31-86d3-bf284b5940da" containerID="bb3bb5acb86024306c440b1b8370735b7f28aee15fcbf9f6cc218a52f686a292" exitCode=0 Jul 02 19:36:19.460761 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:19.460348 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" event={"ID":"a7f638a6-6cb9-4e31-86d3-bf284b5940da","Type":"ContainerDied","Data":"bb3bb5acb86024306c440b1b8370735b7f28aee15fcbf9f6cc218a52f686a292"} Jul 02 19:36:19.460761 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:19.460752 2578 scope.go:117] "RemoveContainer" containerID="bb3bb5acb86024306c440b1b8370735b7f28aee15fcbf9f6cc218a52f686a292" Jul 02 19:36:19.461724 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:19.461696 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:36:19.461826 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:19.461763 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:36:19.461826 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:19.461797 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:36:20.381554 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:20.381516 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:20.467956 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:20.467921 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" event={"ID":"a7f638a6-6cb9-4e31-86d3-bf284b5940da","Type":"ContainerStarted","Data":"8a328a5509f226d744410c8d85e9c5007d09af0b6ac3c51290781c626cbd11c2"} Jul 02 19:36:21.158493 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:21.158444 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:21.160001 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:21.159967 2578 patch_prober.go:28] interesting pod/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 02 19:36:21.160144 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:21.160039 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" podUID="a7f638a6-6cb9-4e31-86d3-bf284b5940da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 02 19:36:22.158783 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:22.158740 2578 patch_prober.go:28] interesting pod/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 02 19:36:22.159174 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:22.158815 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" podUID="a7f638a6-6cb9-4e31-86d3-bf284b5940da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 02 19:36:23.158426 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:23.158356 2578 patch_prober.go:28] interesting pod/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 02 19:36:23.158731 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:23.158700 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" podUID="a7f638a6-6cb9-4e31-86d3-bf284b5940da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 02 19:36:24.159245 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:24.159209 2578 patch_prober.go:28] interesting pod/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 02 19:36:24.159875 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:24.159839 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" podUID="a7f638a6-6cb9-4e31-86d3-bf284b5940da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 02 19:36:25.162799 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:25.162769 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:25.163171 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:25.163153 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:25.163759 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:25.163735 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:25.388413 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:25.388366 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7"] Jul 02 19:36:25.499458 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:25.499364 2578 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" secret="" err="secret \"e2e-mt-2cd582b7-openshift-default-dockercfg-w4fv8\" not found" Jul 02 19:36:26.504621 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:26.504581 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" podUID="a7f638a6-6cb9-4e31-86d3-bf284b5940da" containerName="istio-proxy" containerID="cri-o://8a328a5509f226d744410c8d85e9c5007d09af0b6ac3c51290781c626cbd11c2" gracePeriod=30 Jul 02 19:36:31.763873 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.763850 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:31.872815 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.872785 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-workload-certs\") pod \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " Jul 02 19:36:31.872815 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.872821 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrnt4\" (UniqueName: \"kubernetes.io/projected/a7f638a6-6cb9-4e31-86d3-bf284b5940da-kube-api-access-mrnt4\") pod \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " Jul 02 19:36:31.873070 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.872882 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-credential-socket\") pod \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " Jul 02 19:36:31.873070 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.872898 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-token\") pod \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " Jul 02 19:36:31.873070 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.872918 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-podinfo\") pod \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " Jul 02 19:36:31.873070 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.872932 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istiod-ca-cert\") pod \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " Jul 02 19:36:31.873070 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.872971 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-workload-socket\") pod \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " Jul 02 19:36:31.873070 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.873009 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-data\") pod \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " Jul 02 19:36:31.873070 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.873051 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-envoy\") pod \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\" (UID: \"a7f638a6-6cb9-4e31-86d3-bf284b5940da\") " Jul 02 19:36:31.873532 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.873117 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a7f638a6-6cb9-4e31-86d3-bf284b5940da" (UID: "a7f638a6-6cb9-4e31-86d3-bf284b5940da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:36:31.873532 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.873239 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a7f638a6-6cb9-4e31-86d3-bf284b5940da" (UID: "a7f638a6-6cb9-4e31-86d3-bf284b5940da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:36:31.873532 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.873453 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a7f638a6-6cb9-4e31-86d3-bf284b5940da" (UID: "a7f638a6-6cb9-4e31-86d3-bf284b5940da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:36:31.873532 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.873481 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-credential-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:36:31.873532 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.873504 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-workload-certs\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:36:31.873744 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.873631 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a7f638a6-6cb9-4e31-86d3-bf284b5940da" (UID: "a7f638a6-6cb9-4e31-86d3-bf284b5940da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:36:31.874260 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.874237 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-data" (OuterVolumeSpecName: "istio-data") pod "a7f638a6-6cb9-4e31-86d3-bf284b5940da" (UID: "a7f638a6-6cb9-4e31-86d3-bf284b5940da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:36:31.875323 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.875300 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-token" (OuterVolumeSpecName: "istio-token") pod "a7f638a6-6cb9-4e31-86d3-bf284b5940da" (UID: "a7f638a6-6cb9-4e31-86d3-bf284b5940da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:36:31.875461 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.875447 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a7f638a6-6cb9-4e31-86d3-bf284b5940da" (UID: "a7f638a6-6cb9-4e31-86d3-bf284b5940da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:36:31.875666 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.875647 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a7f638a6-6cb9-4e31-86d3-bf284b5940da" (UID: "a7f638a6-6cb9-4e31-86d3-bf284b5940da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:36:31.875714 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.875670 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7f638a6-6cb9-4e31-86d3-bf284b5940da-kube-api-access-mrnt4" (OuterVolumeSpecName: "kube-api-access-mrnt4") pod "a7f638a6-6cb9-4e31-86d3-bf284b5940da" (UID: "a7f638a6-6cb9-4e31-86d3-bf284b5940da"). InnerVolumeSpecName "kube-api-access-mrnt4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:36:31.974765 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.974732 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-workload-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:36:31.974765 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.974758 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-data\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:36:31.974765 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.974767 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-envoy\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:36:31.974988 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.974775 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrnt4\" (UniqueName: \"kubernetes.io/projected/a7f638a6-6cb9-4e31-86d3-bf284b5940da-kube-api-access-mrnt4\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:36:31.974988 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.974784 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-token\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:36:31.974988 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.974793 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istio-podinfo\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:36:31.974988 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:31.974800 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7f638a6-6cb9-4e31-86d3-bf284b5940da-istiod-ca-cert\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:36:32.544413 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:32.544356 2578 generic.go:356] "Generic (PLEG): container finished" podID="a7f638a6-6cb9-4e31-86d3-bf284b5940da" containerID="8a328a5509f226d744410c8d85e9c5007d09af0b6ac3c51290781c626cbd11c2" exitCode=0 Jul 02 19:36:32.544413 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:32.544408 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" event={"ID":"a7f638a6-6cb9-4e31-86d3-bf284b5940da","Type":"ContainerDied","Data":"8a328a5509f226d744410c8d85e9c5007d09af0b6ac3c51290781c626cbd11c2"} Jul 02 19:36:32.544670 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:32.544421 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" Jul 02 19:36:32.544670 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:32.544451 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7" event={"ID":"a7f638a6-6cb9-4e31-86d3-bf284b5940da","Type":"ContainerDied","Data":"849bc38f31c4593e346c60b07623117b54ac6901f64242e18df0b53113689c88"} Jul 02 19:36:32.544670 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:32.544474 2578 scope.go:117] "RemoveContainer" containerID="8a328a5509f226d744410c8d85e9c5007d09af0b6ac3c51290781c626cbd11c2" Jul 02 19:36:32.553821 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:32.553804 2578 scope.go:117] "RemoveContainer" containerID="bb3bb5acb86024306c440b1b8370735b7f28aee15fcbf9f6cc218a52f686a292" Jul 02 19:36:32.561629 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:32.561612 2578 scope.go:117] "RemoveContainer" containerID="8a328a5509f226d744410c8d85e9c5007d09af0b6ac3c51290781c626cbd11c2" Jul 02 19:36:32.561881 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:36:32.561863 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a328a5509f226d744410c8d85e9c5007d09af0b6ac3c51290781c626cbd11c2\": container with ID starting with 8a328a5509f226d744410c8d85e9c5007d09af0b6ac3c51290781c626cbd11c2 not found: ID does not exist" containerID="8a328a5509f226d744410c8d85e9c5007d09af0b6ac3c51290781c626cbd11c2" Jul 02 19:36:32.561945 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:32.561889 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a328a5509f226d744410c8d85e9c5007d09af0b6ac3c51290781c626cbd11c2"} err="failed to get container status \"8a328a5509f226d744410c8d85e9c5007d09af0b6ac3c51290781c626cbd11c2\": rpc error: code = NotFound desc = could not find container \"8a328a5509f226d744410c8d85e9c5007d09af0b6ac3c51290781c626cbd11c2\": container with ID starting with 8a328a5509f226d744410c8d85e9c5007d09af0b6ac3c51290781c626cbd11c2 not found: ID does not exist" Jul 02 19:36:32.561945 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:32.561906 2578 scope.go:117] "RemoveContainer" containerID="bb3bb5acb86024306c440b1b8370735b7f28aee15fcbf9f6cc218a52f686a292" Jul 02 19:36:32.562131 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:36:32.562115 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb3bb5acb86024306c440b1b8370735b7f28aee15fcbf9f6cc218a52f686a292\": container with ID starting with bb3bb5acb86024306c440b1b8370735b7f28aee15fcbf9f6cc218a52f686a292 not found: ID does not exist" containerID="bb3bb5acb86024306c440b1b8370735b7f28aee15fcbf9f6cc218a52f686a292" Jul 02 19:36:32.562173 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:32.562138 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb3bb5acb86024306c440b1b8370735b7f28aee15fcbf9f6cc218a52f686a292"} err="failed to get container status \"bb3bb5acb86024306c440b1b8370735b7f28aee15fcbf9f6cc218a52f686a292\": rpc error: code = NotFound desc = could not find container \"bb3bb5acb86024306c440b1b8370735b7f28aee15fcbf9f6cc218a52f686a292\": container with ID starting with bb3bb5acb86024306c440b1b8370735b7f28aee15fcbf9f6cc218a52f686a292 not found: ID does not exist" Jul 02 19:36:32.567835 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:32.567815 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7"] Jul 02 19:36:32.572496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:32.572474 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2cd582b7-openshift-default-76649d44b7-cs4x7"] Jul 02 19:36:33.727202 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:33.727167 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7f638a6-6cb9-4e31-86d3-bf284b5940da" path="/var/lib/kubelet/pods/a7f638a6-6cb9-4e31-86d3-bf284b5940da/volumes" Jul 02 19:36:45.108868 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.108835 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm"] Jul 02 19:36:45.116486 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.109670 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7f638a6-6cb9-4e31-86d3-bf284b5940da" containerName="istio-proxy" Jul 02 19:36:45.116486 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.109701 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7f638a6-6cb9-4e31-86d3-bf284b5940da" containerName="istio-proxy" Jul 02 19:36:45.116486 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.109747 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7f638a6-6cb9-4e31-86d3-bf284b5940da" containerName="istio-proxy" Jul 02 19:36:45.116486 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.109757 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7f638a6-6cb9-4e31-86d3-bf284b5940da" containerName="istio-proxy" Jul 02 19:36:45.116486 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.110028 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7f638a6-6cb9-4e31-86d3-bf284b5940da" containerName="istio-proxy" Jul 02 19:36:45.116486 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.110041 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7f638a6-6cb9-4e31-86d3-bf284b5940da" containerName="istio-proxy" Jul 02 19:36:45.116486 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.114095 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.119489 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.119461 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a603bec7-openshift-default-dockercfg-k2nm8\"" Jul 02 19:36:45.130042 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.130015 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm"] Jul 02 19:36:45.193273 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.193237 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-workload-certs\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.193467 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.193302 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-envoy\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.193467 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.193355 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-workload-socket\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.193581 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.193476 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-credential-socket\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.193581 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.193528 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-token\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.193581 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.193556 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-podinfo\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.193734 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.193590 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-data\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.193734 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.193649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1b9662e-a289-4344-adde-39360ab0a8e7-istiod-ca-cert\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.193734 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.193684 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9nf2\" (UniqueName: \"kubernetes.io/projected/c1b9662e-a289-4344-adde-39360ab0a8e7-kube-api-access-n9nf2\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.294487 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.294456 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1b9662e-a289-4344-adde-39360ab0a8e7-istiod-ca-cert\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.294487 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.294490 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9nf2\" (UniqueName: \"kubernetes.io/projected/c1b9662e-a289-4344-adde-39360ab0a8e7-kube-api-access-n9nf2\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.294710 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.294520 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-workload-certs\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.294710 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.294548 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-envoy\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.294710 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.294579 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-workload-socket\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.294869 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.294718 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-credential-socket\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.294869 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.294767 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-token\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.294869 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.294800 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-podinfo\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.294869 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.294837 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-data\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.295076 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.295056 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-workload-socket\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.295138 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.295105 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-workload-certs\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.295380 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.295354 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-data\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.295511 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.295436 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-credential-socket\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.295739 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.295713 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1b9662e-a289-4344-adde-39360ab0a8e7-istiod-ca-cert\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.297327 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.297303 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-envoy\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.298368 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.298344 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-podinfo\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.303705 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.303681 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-token\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.303905 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.303886 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9nf2\" (UniqueName: \"kubernetes.io/projected/c1b9662e-a289-4344-adde-39360ab0a8e7-kube-api-access-n9nf2\") pod \"e2e-mt-a603bec7-openshift-default-7c86995467-67scm\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.426958 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.426869 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:45.591494 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.591471 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm"] Jul 02 19:36:45.594255 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:36:45.594228 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1b9662e_a289_4344_adde_39360ab0a8e7.slice/crio-73890949cca72d5bb0100c5ddb1bdb5e7eebbb574fab2a78f17123b8a4531020 WatchSource:0}: Error finding container 73890949cca72d5bb0100c5ddb1bdb5e7eebbb574fab2a78f17123b8a4531020: Status 404 returned error can't find the container with id 73890949cca72d5bb0100c5ddb1bdb5e7eebbb574fab2a78f17123b8a4531020 Jul 02 19:36:45.596995 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.596962 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:36:45.597114 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.597035 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:36:45.597114 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.597084 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:36:45.618355 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:45.618323 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" event={"ID":"c1b9662e-a289-4344-adde-39360ab0a8e7","Type":"ContainerStarted","Data":"73890949cca72d5bb0100c5ddb1bdb5e7eebbb574fab2a78f17123b8a4531020"} Jul 02 19:36:46.625231 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:46.625187 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" event={"ID":"c1b9662e-a289-4344-adde-39360ab0a8e7","Type":"ContainerStarted","Data":"2888cd16211ea78ccfad85c04eaad784645097e38eff45a0d7a13a29deb818cf"} Jul 02 19:36:47.427446 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:47.427412 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:47.432347 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:47.432323 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:47.453082 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:47.453006 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" podStartSLOduration=2.452990351 podStartE2EDuration="2.452990351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:36:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:36:46.65094634 +0000 UTC m=+1749.453761802" watchObservedRunningTime="2026-07-02 19:36:47.452990351 +0000 UTC m=+1750.255805801" Jul 02 19:36:47.630447 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:47.630415 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:36:47.631464 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:36:47.631445 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:37:02.631778 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:02.631740 2578 patch_prober.go:28] interesting pod/e2e-mt-a603bec7-openshift-default-7c86995467-67scm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 02 19:37:02.632274 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:02.631820 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" podUID="c1b9662e-a289-4344-adde-39360ab0a8e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 02 19:37:02.712213 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:02.712180 2578 generic.go:356] "Generic (PLEG): container finished" podID="c1b9662e-a289-4344-adde-39360ab0a8e7" containerID="2888cd16211ea78ccfad85c04eaad784645097e38eff45a0d7a13a29deb818cf" exitCode=0 Jul 02 19:37:02.712378 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:02.712258 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" event={"ID":"c1b9662e-a289-4344-adde-39360ab0a8e7","Type":"ContainerDied","Data":"2888cd16211ea78ccfad85c04eaad784645097e38eff45a0d7a13a29deb818cf"} Jul 02 19:37:02.712775 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:02.712759 2578 scope.go:117] "RemoveContainer" containerID="2888cd16211ea78ccfad85c04eaad784645097e38eff45a0d7a13a29deb818cf" Jul 02 19:37:02.713754 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:02.713728 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:37:02.713811 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:02.713791 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:37:02.713857 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:02.713822 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:37:03.726494 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:03.726461 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" event={"ID":"c1b9662e-a289-4344-adde-39360ab0a8e7","Type":"ContainerStarted","Data":"3c70c73918deacc99b4e53def153644762b0f365239a2bf110fb11410982632d"} Jul 02 19:37:04.427116 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:04.427080 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:37:04.427430 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:04.427298 2578 patch_prober.go:28] interesting pod/e2e-mt-a603bec7-openshift-default-7c86995467-67scm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 02 19:37:04.427430 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:04.427345 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" podUID="c1b9662e-a289-4344-adde-39360ab0a8e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 02 19:37:05.427653 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:05.427618 2578 patch_prober.go:28] interesting pod/e2e-mt-a603bec7-openshift-default-7c86995467-67scm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 02 19:37:05.428043 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:05.427680 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" podUID="c1b9662e-a289-4344-adde-39360ab0a8e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 02 19:37:06.427548 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:06.427512 2578 patch_prober.go:28] interesting pod/e2e-mt-a603bec7-openshift-default-7c86995467-67scm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 02 19:37:06.427722 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:06.427581 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" podUID="c1b9662e-a289-4344-adde-39360ab0a8e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 02 19:37:07.427512 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:07.427479 2578 patch_prober.go:28] interesting pod/e2e-mt-a603bec7-openshift-default-7c86995467-67scm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 02 19:37:07.427695 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:07.427532 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" podUID="c1b9662e-a289-4344-adde-39360ab0a8e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 02 19:37:08.431179 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:08.431148 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:37:08.431680 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:08.431500 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:37:08.432147 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:37:08.432130 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:38:25.026183 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:25.026146 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm"] Jul 02 19:38:25.026655 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:25.026360 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" podUID="c1b9662e-a289-4344-adde-39360ab0a8e7" containerName="istio-proxy" containerID="cri-o://3c70c73918deacc99b4e53def153644762b0f365239a2bf110fb11410982632d" gracePeriod=30 Jul 02 19:38:30.186185 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.186145 2578 generic.go:356] "Generic (PLEG): container finished" podID="c1b9662e-a289-4344-adde-39360ab0a8e7" containerID="3c70c73918deacc99b4e53def153644762b0f365239a2bf110fb11410982632d" exitCode=0 Jul 02 19:38:30.186649 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.186279 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" event={"ID":"c1b9662e-a289-4344-adde-39360ab0a8e7","Type":"ContainerDied","Data":"3c70c73918deacc99b4e53def153644762b0f365239a2bf110fb11410982632d"} Jul 02 19:38:30.186649 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.186334 2578 scope.go:117] "RemoveContainer" containerID="2888cd16211ea78ccfad85c04eaad784645097e38eff45a0d7a13a29deb818cf" Jul 02 19:38:30.297978 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.297912 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:38:30.405827 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.405791 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-workload-socket\") pod \"c1b9662e-a289-4344-adde-39360ab0a8e7\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " Jul 02 19:38:30.406012 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.405843 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-workload-certs\") pod \"c1b9662e-a289-4344-adde-39360ab0a8e7\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " Jul 02 19:38:30.406012 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.405897 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-credential-socket\") pod \"c1b9662e-a289-4344-adde-39360ab0a8e7\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " Jul 02 19:38:30.406012 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.405921 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-token\") pod \"c1b9662e-a289-4344-adde-39360ab0a8e7\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " Jul 02 19:38:30.406012 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.405948 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1b9662e-a289-4344-adde-39360ab0a8e7-istiod-ca-cert\") pod \"c1b9662e-a289-4344-adde-39360ab0a8e7\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " Jul 02 19:38:30.406226 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.406043 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9nf2\" (UniqueName: \"kubernetes.io/projected/c1b9662e-a289-4344-adde-39360ab0a8e7-kube-api-access-n9nf2\") pod \"c1b9662e-a289-4344-adde-39360ab0a8e7\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " Jul 02 19:38:30.406226 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.406077 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c1b9662e-a289-4344-adde-39360ab0a8e7" (UID: "c1b9662e-a289-4344-adde-39360ab0a8e7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:38:30.406226 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.406166 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c1b9662e-a289-4344-adde-39360ab0a8e7" (UID: "c1b9662e-a289-4344-adde-39360ab0a8e7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:38:30.406226 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.406191 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c1b9662e-a289-4344-adde-39360ab0a8e7" (UID: "c1b9662e-a289-4344-adde-39360ab0a8e7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:38:30.406529 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.406349 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1b9662e-a289-4344-adde-39360ab0a8e7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c1b9662e-a289-4344-adde-39360ab0a8e7" (UID: "c1b9662e-a289-4344-adde-39360ab0a8e7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:38:30.406529 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.406420 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-envoy\") pod \"c1b9662e-a289-4344-adde-39360ab0a8e7\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " Jul 02 19:38:30.406529 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.406446 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-podinfo\") pod \"c1b9662e-a289-4344-adde-39360ab0a8e7\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " Jul 02 19:38:30.406529 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.406467 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-data\") pod \"c1b9662e-a289-4344-adde-39360ab0a8e7\" (UID: \"c1b9662e-a289-4344-adde-39360ab0a8e7\") " Jul 02 19:38:30.407297 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.407242 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-data" (OuterVolumeSpecName: "istio-data") pod "c1b9662e-a289-4344-adde-39360ab0a8e7" (UID: "c1b9662e-a289-4344-adde-39360ab0a8e7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:38:30.407714 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.407692 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-workload-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:38:30.407809 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.407721 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-workload-certs\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:38:30.407809 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.407737 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-credential-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:38:30.407809 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.407750 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1b9662e-a289-4344-adde-39360ab0a8e7-istiod-ca-cert\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:38:30.407809 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.407762 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-data\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:38:30.408661 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.408633 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c1b9662e-a289-4344-adde-39360ab0a8e7" (UID: "c1b9662e-a289-4344-adde-39360ab0a8e7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:38:30.408776 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.408707 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-token" (OuterVolumeSpecName: "istio-token") pod "c1b9662e-a289-4344-adde-39360ab0a8e7" (UID: "c1b9662e-a289-4344-adde-39360ab0a8e7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:38:30.408968 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.408931 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1b9662e-a289-4344-adde-39360ab0a8e7-kube-api-access-n9nf2" (OuterVolumeSpecName: "kube-api-access-n9nf2") pod "c1b9662e-a289-4344-adde-39360ab0a8e7" (UID: "c1b9662e-a289-4344-adde-39360ab0a8e7"). InnerVolumeSpecName "kube-api-access-n9nf2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:38:30.409045 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.409027 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c1b9662e-a289-4344-adde-39360ab0a8e7" (UID: "c1b9662e-a289-4344-adde-39360ab0a8e7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:38:30.508796 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.508761 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-token\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:38:30.508796 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.508791 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9nf2\" (UniqueName: \"kubernetes.io/projected/c1b9662e-a289-4344-adde-39360ab0a8e7-kube-api-access-n9nf2\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:38:30.508796 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.508804 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-envoy\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:38:30.509028 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:30.508815 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1b9662e-a289-4344-adde-39360ab0a8e7-istio-podinfo\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:38:31.192741 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:31.192713 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" Jul 02 19:38:31.193180 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:31.192746 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm" event={"ID":"c1b9662e-a289-4344-adde-39360ab0a8e7","Type":"ContainerDied","Data":"73890949cca72d5bb0100c5ddb1bdb5e7eebbb574fab2a78f17123b8a4531020"} Jul 02 19:38:31.193180 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:31.192806 2578 scope.go:117] "RemoveContainer" containerID="3c70c73918deacc99b4e53def153644762b0f365239a2bf110fb11410982632d" Jul 02 19:38:31.214582 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:31.214555 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm"] Jul 02 19:38:31.219273 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:31.219253 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a603bec7-openshift-default-7c86995467-67scm"] Jul 02 19:38:31.727519 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:38:31.727485 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1b9662e-a289-4344-adde-39360ab0a8e7" path="/var/lib/kubelet/pods/c1b9662e-a289-4344-adde-39360ab0a8e7/volumes" Jul 02 19:39:20.328449 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.328415 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9"] Jul 02 19:39:20.328851 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.328767 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1b9662e-a289-4344-adde-39360ab0a8e7" containerName="istio-proxy" Jul 02 19:39:20.328851 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.328778 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1b9662e-a289-4344-adde-39360ab0a8e7" containerName="istio-proxy" Jul 02 19:39:20.328851 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.328788 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1b9662e-a289-4344-adde-39360ab0a8e7" containerName="istio-proxy" Jul 02 19:39:20.328851 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.328794 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1b9662e-a289-4344-adde-39360ab0a8e7" containerName="istio-proxy" Jul 02 19:39:20.328978 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.328860 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1b9662e-a289-4344-adde-39360ab0a8e7" containerName="istio-proxy" Jul 02 19:39:20.328978 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.328867 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1b9662e-a289-4344-adde-39360ab0a8e7" containerName="istio-proxy" Jul 02 19:39:20.331957 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.331939 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.334409 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.334383 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-09f1f6-openshift-default-dockercfg-f6r6m\"" Jul 02 19:39:20.344657 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.344639 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9"] Jul 02 19:39:20.421264 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.421218 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbtss\" (UniqueName: \"kubernetes.io/projected/8ba6d4de-14e9-4bb9-b896-21b693858de1-kube-api-access-jbtss\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.421264 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.421266 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-envoy\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.421523 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.421330 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-token\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.421523 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.421368 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-workload-certs\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.421523 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.421427 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-podinfo\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.421523 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.421472 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-workload-socket\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.421523 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.421486 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ba6d4de-14e9-4bb9-b896-21b693858de1-istiod-ca-cert\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.421523 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.421526 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-credential-socket\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.421850 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.421544 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-data\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.522628 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.522595 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-credential-socket\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.522628 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.522631 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-data\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.522828 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.522661 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbtss\" (UniqueName: \"kubernetes.io/projected/8ba6d4de-14e9-4bb9-b896-21b693858de1-kube-api-access-jbtss\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.522828 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.522694 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-envoy\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.522828 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.522746 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-token\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.522828 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.522786 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-workload-certs\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.522828 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.522806 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-podinfo\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.523082 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.523042 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-data\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.523142 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.523095 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-credential-socket\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.523186 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.523145 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-workload-socket\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.523229 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.523185 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ba6d4de-14e9-4bb9-b896-21b693858de1-istiod-ca-cert\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.523279 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.523245 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-workload-certs\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.523318 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.523299 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-workload-socket\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.523740 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.523725 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ba6d4de-14e9-4bb9-b896-21b693858de1-istiod-ca-cert\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.525123 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.525095 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-envoy\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.525521 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.525498 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-podinfo\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.533168 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.533145 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-token\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.533516 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.533490 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbtss\" (UniqueName: \"kubernetes.io/projected/8ba6d4de-14e9-4bb9-b896-21b693858de1-kube-api-access-jbtss\") pod \"e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.645495 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.645420 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:20.787031 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.787009 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9"] Jul 02 19:39:20.788881 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:39:20.788843 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ba6d4de_14e9_4bb9_b896_21b693858de1.slice/crio-698fa9fe8d7821ca27d35561842e1ed7e33f69193c52a95de619ac4f5bf34871 WatchSource:0}: Error finding container 698fa9fe8d7821ca27d35561842e1ed7e33f69193c52a95de619ac4f5bf34871: Status 404 returned error can't find the container with id 698fa9fe8d7821ca27d35561842e1ed7e33f69193c52a95de619ac4f5bf34871 Jul 02 19:39:20.791010 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.790966 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:39:20.791071 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.791056 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:39:20.791117 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:20.791102 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:39:21.459102 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:21.459071 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" event={"ID":"8ba6d4de-14e9-4bb9-b896-21b693858de1","Type":"ContainerStarted","Data":"bbf2e89c263906fb29c33ee998859ad88211af258b058c5de01bd93863afa18f"} Jul 02 19:39:21.459516 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:21.459121 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" event={"ID":"8ba6d4de-14e9-4bb9-b896-21b693858de1","Type":"ContainerStarted","Data":"698fa9fe8d7821ca27d35561842e1ed7e33f69193c52a95de619ac4f5bf34871"} Jul 02 19:39:21.482361 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:21.482313 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" podStartSLOduration=1.482294053 podStartE2EDuration="1.482294053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:39:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:39:21.481826841 +0000 UTC m=+1904.284642323" watchObservedRunningTime="2026-07-02 19:39:21.482294053 +0000 UTC m=+1904.285109502" Jul 02 19:39:21.646557 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:21.646526 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:21.651133 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:21.651109 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:22.465183 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:22.465155 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:22.466013 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:22.465996 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:28.706725 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.706689 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc"] Jul 02 19:39:28.710571 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.710550 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.713161 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.713137 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-cc922a-openshift-default-dockercfg-76k5r\"" Jul 02 19:39:28.724545 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.724522 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc"] Jul 02 19:39:28.797362 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.797336 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-workload-certs\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.797518 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.797381 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-workload-socket\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.797518 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.797431 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-podinfo\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.797518 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.797472 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istiod-ca-cert\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.797518 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.797491 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-credential-socket\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.797518 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.797506 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-envoy\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.797702 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.797550 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-data\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.797702 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.797587 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-token\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.797702 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.797644 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5mjn\" (UniqueName: \"kubernetes.io/projected/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-kube-api-access-z5mjn\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.898390 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.898350 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5mjn\" (UniqueName: \"kubernetes.io/projected/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-kube-api-access-z5mjn\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.898390 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.898415 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-workload-certs\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.898656 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.898446 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-workload-socket\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.898656 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.898461 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-podinfo\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.898656 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.898483 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istiod-ca-cert\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.898656 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.898503 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-credential-socket\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.898656 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.898525 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-envoy\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.898656 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.898581 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-data\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.898656 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.898604 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-token\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.899054 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.898915 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-workload-socket\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.899054 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.898971 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-workload-certs\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.899216 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.899193 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-data\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.899383 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.899221 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-credential-socket\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.899495 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.899287 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istiod-ca-cert\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.901121 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.901098 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-envoy\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.901247 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.901217 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-podinfo\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.907368 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.907341 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-token\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:28.907763 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:28.907743 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5mjn\" (UniqueName: \"kubernetes.io/projected/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-kube-api-access-z5mjn\") pod \"e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:29.021348 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:29.021271 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:29.163170 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:29.163144 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc"] Jul 02 19:39:29.165366 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:39:29.165339 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcdcdff0_49b1_4d33_95b2_2b66c07d5fae.slice/crio-947c1a77a47a0855b5290d47761e05d4481c17141a088979d63c097856106b4e WatchSource:0}: Error finding container 947c1a77a47a0855b5290d47761e05d4481c17141a088979d63c097856106b4e: Status 404 returned error can't find the container with id 947c1a77a47a0855b5290d47761e05d4481c17141a088979d63c097856106b4e Jul 02 19:39:29.167838 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:29.167764 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:39:29.167929 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:29.167890 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:39:29.167979 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:29.167936 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:39:29.508143 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:29.508105 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" event={"ID":"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae","Type":"ContainerStarted","Data":"240f26cb18d1188c39f159f40f1cbd06cf6a5304a4bd33988902744ecd285d3f"} Jul 02 19:39:29.508333 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:29.508202 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" event={"ID":"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae","Type":"ContainerStarted","Data":"947c1a77a47a0855b5290d47761e05d4481c17141a088979d63c097856106b4e"} Jul 02 19:39:29.533508 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:29.533457 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" podStartSLOduration=1.533442508 podStartE2EDuration="1.533442508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:39:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:39:29.530213601 +0000 UTC m=+1912.333029053" watchObservedRunningTime="2026-07-02 19:39:29.533442508 +0000 UTC m=+1912.336257958" Jul 02 19:39:30.022342 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:30.022310 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:30.027689 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:30.027649 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:30.513609 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:30.513577 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:30.514383 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:30.514367 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:38.506575 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:38.506538 2578 patch_prober.go:28] interesting pod/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 19:39:38.507023 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:38.506608 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" podUID="8ba6d4de-14e9-4bb9-b896-21b693858de1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 19:39:39.574471 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:39.574441 2578 generic.go:356] "Generic (PLEG): container finished" podID="8ba6d4de-14e9-4bb9-b896-21b693858de1" containerID="bbf2e89c263906fb29c33ee998859ad88211af258b058c5de01bd93863afa18f" exitCode=0 Jul 02 19:39:39.574989 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:39.574525 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" event={"ID":"8ba6d4de-14e9-4bb9-b896-21b693858de1","Type":"ContainerDied","Data":"bbf2e89c263906fb29c33ee998859ad88211af258b058c5de01bd93863afa18f"} Jul 02 19:39:39.575114 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:39.575082 2578 scope.go:117] "RemoveContainer" containerID="bbf2e89c263906fb29c33ee998859ad88211af258b058c5de01bd93863afa18f" Jul 02 19:39:39.576237 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:39.576205 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:39:39.576341 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:39.576288 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:39:39.576341 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:39.576324 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:39:40.595851 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:40.595809 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" event={"ID":"8ba6d4de-14e9-4bb9-b896-21b693858de1","Type":"ContainerStarted","Data":"0c68a4d068f3103afcbf45f820f439d1274b3f879b409e56c935ade4c426d071"} Jul 02 19:39:40.645811 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:40.645765 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:40.646280 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:40.646249 2578 patch_prober.go:28] interesting pod/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 02 19:39:40.646386 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:40.646319 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" podUID="8ba6d4de-14e9-4bb9-b896-21b693858de1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 02 19:39:41.646945 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:41.646907 2578 patch_prober.go:28] interesting pod/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 02 19:39:41.647559 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:41.647525 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" podUID="8ba6d4de-14e9-4bb9-b896-21b693858de1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 02 19:39:42.646844 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:42.646806 2578 patch_prober.go:28] interesting pod/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 02 19:39:42.647156 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:42.647126 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" podUID="8ba6d4de-14e9-4bb9-b896-21b693858de1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 02 19:39:43.646184 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:43.646143 2578 patch_prober.go:28] interesting pod/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 02 19:39:43.646660 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:43.646629 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" podUID="8ba6d4de-14e9-4bb9-b896-21b693858de1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 02 19:39:44.649655 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:44.649599 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:44.650294 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:44.650269 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:44.650995 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:44.650974 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:39:46.554568 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:46.554520 2578 patch_prober.go:28] interesting pod/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 19:39:46.555059 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:46.554574 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" podUID="fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 19:39:47.635724 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:47.635687 2578 generic.go:356] "Generic (PLEG): container finished" podID="fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" containerID="240f26cb18d1188c39f159f40f1cbd06cf6a5304a4bd33988902744ecd285d3f" exitCode=0 Jul 02 19:39:47.636255 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:47.635728 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" event={"ID":"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae","Type":"ContainerDied","Data":"240f26cb18d1188c39f159f40f1cbd06cf6a5304a4bd33988902744ecd285d3f"} Jul 02 19:39:47.636255 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:47.636249 2578 scope.go:117] "RemoveContainer" containerID="240f26cb18d1188c39f159f40f1cbd06cf6a5304a4bd33988902744ecd285d3f" Jul 02 19:39:47.637517 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:47.637481 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:39:47.637619 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:47.637570 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:39:47.637619 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:47.637612 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:39:48.646072 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:48.646020 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" event={"ID":"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae","Type":"ContainerStarted","Data":"0359658e7af98147ea875c14878f726eea68b451403859d42f3c41f942dbd33c"} Jul 02 19:39:49.022213 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:49.022128 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:49.022369 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:49.022317 2578 patch_prober.go:28] interesting pod/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 02 19:39:49.022439 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:49.022361 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" podUID="fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 02 19:39:50.022104 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:50.022074 2578 patch_prober.go:28] interesting pod/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 02 19:39:50.022536 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:50.022122 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" podUID="fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 02 19:39:51.022391 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:51.022352 2578 patch_prober.go:28] interesting pod/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 02 19:39:51.022863 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:51.022443 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" podUID="fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 02 19:39:52.022514 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:52.022482 2578 patch_prober.go:28] interesting pod/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 02 19:39:52.022898 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:52.022534 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" podUID="fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 02 19:39:53.025362 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:53.025335 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:53.025787 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:53.025615 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:39:53.026355 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:39:53.026337 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:41:33.955224 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:33.955193 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9"] Jul 02 19:41:33.955766 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:33.955673 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" podUID="8ba6d4de-14e9-4bb9-b896-21b693858de1" containerName="istio-proxy" containerID="cri-o://0c68a4d068f3103afcbf45f820f439d1274b3f879b409e56c935ade4c426d071" gracePeriod=30 Jul 02 19:41:39.214657 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.214631 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:41:39.244648 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.244570 2578 generic.go:356] "Generic (PLEG): container finished" podID="8ba6d4de-14e9-4bb9-b896-21b693858de1" containerID="0c68a4d068f3103afcbf45f820f439d1274b3f879b409e56c935ade4c426d071" exitCode=0 Jul 02 19:41:39.244648 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.244612 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" Jul 02 19:41:39.244846 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.244647 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" event={"ID":"8ba6d4de-14e9-4bb9-b896-21b693858de1","Type":"ContainerDied","Data":"0c68a4d068f3103afcbf45f820f439d1274b3f879b409e56c935ade4c426d071"} Jul 02 19:41:39.244846 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.244714 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9" event={"ID":"8ba6d4de-14e9-4bb9-b896-21b693858de1","Type":"ContainerDied","Data":"698fa9fe8d7821ca27d35561842e1ed7e33f69193c52a95de619ac4f5bf34871"} Jul 02 19:41:39.244846 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.244745 2578 scope.go:117] "RemoveContainer" containerID="0c68a4d068f3103afcbf45f820f439d1274b3f879b409e56c935ade4c426d071" Jul 02 19:41:39.253637 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.253621 2578 scope.go:117] "RemoveContainer" containerID="bbf2e89c263906fb29c33ee998859ad88211af258b058c5de01bd93863afa18f" Jul 02 19:41:39.261883 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.261862 2578 scope.go:117] "RemoveContainer" containerID="0c68a4d068f3103afcbf45f820f439d1274b3f879b409e56c935ade4c426d071" Jul 02 19:41:39.262175 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:41:39.262156 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c68a4d068f3103afcbf45f820f439d1274b3f879b409e56c935ade4c426d071\": container with ID starting with 0c68a4d068f3103afcbf45f820f439d1274b3f879b409e56c935ade4c426d071 not found: ID does not exist" containerID="0c68a4d068f3103afcbf45f820f439d1274b3f879b409e56c935ade4c426d071" Jul 02 19:41:39.262227 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.262184 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c68a4d068f3103afcbf45f820f439d1274b3f879b409e56c935ade4c426d071"} err="failed to get container status \"0c68a4d068f3103afcbf45f820f439d1274b3f879b409e56c935ade4c426d071\": rpc error: code = NotFound desc = could not find container \"0c68a4d068f3103afcbf45f820f439d1274b3f879b409e56c935ade4c426d071\": container with ID starting with 0c68a4d068f3103afcbf45f820f439d1274b3f879b409e56c935ade4c426d071 not found: ID does not exist" Jul 02 19:41:39.262227 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.262201 2578 scope.go:117] "RemoveContainer" containerID="bbf2e89c263906fb29c33ee998859ad88211af258b058c5de01bd93863afa18f" Jul 02 19:41:39.262465 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:41:39.262449 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbf2e89c263906fb29c33ee998859ad88211af258b058c5de01bd93863afa18f\": container with ID starting with bbf2e89c263906fb29c33ee998859ad88211af258b058c5de01bd93863afa18f not found: ID does not exist" containerID="bbf2e89c263906fb29c33ee998859ad88211af258b058c5de01bd93863afa18f" Jul 02 19:41:39.262515 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.262471 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbf2e89c263906fb29c33ee998859ad88211af258b058c5de01bd93863afa18f"} err="failed to get container status \"bbf2e89c263906fb29c33ee998859ad88211af258b058c5de01bd93863afa18f\": rpc error: code = NotFound desc = could not find container \"bbf2e89c263906fb29c33ee998859ad88211af258b058c5de01bd93863afa18f\": container with ID starting with bbf2e89c263906fb29c33ee998859ad88211af258b058c5de01bd93863afa18f not found: ID does not exist" Jul 02 19:41:39.294876 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.294858 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-podinfo\") pod \"8ba6d4de-14e9-4bb9-b896-21b693858de1\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " Jul 02 19:41:39.294972 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.294919 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbtss\" (UniqueName: \"kubernetes.io/projected/8ba6d4de-14e9-4bb9-b896-21b693858de1-kube-api-access-jbtss\") pod \"8ba6d4de-14e9-4bb9-b896-21b693858de1\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " Jul 02 19:41:39.294972 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.294944 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ba6d4de-14e9-4bb9-b896-21b693858de1-istiod-ca-cert\") pod \"8ba6d4de-14e9-4bb9-b896-21b693858de1\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " Jul 02 19:41:39.294972 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.294966 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-credential-socket\") pod \"8ba6d4de-14e9-4bb9-b896-21b693858de1\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " Jul 02 19:41:39.295110 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.295001 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-data\") pod \"8ba6d4de-14e9-4bb9-b896-21b693858de1\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " Jul 02 19:41:39.295110 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.295028 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-token\") pod \"8ba6d4de-14e9-4bb9-b896-21b693858de1\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " Jul 02 19:41:39.295110 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.295052 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-workload-socket\") pod \"8ba6d4de-14e9-4bb9-b896-21b693858de1\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " Jul 02 19:41:39.295110 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.295096 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-envoy\") pod \"8ba6d4de-14e9-4bb9-b896-21b693858de1\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " Jul 02 19:41:39.295313 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.295135 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-workload-certs\") pod \"8ba6d4de-14e9-4bb9-b896-21b693858de1\" (UID: \"8ba6d4de-14e9-4bb9-b896-21b693858de1\") " Jul 02 19:41:39.295313 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.295271 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8ba6d4de-14e9-4bb9-b896-21b693858de1" (UID: "8ba6d4de-14e9-4bb9-b896-21b693858de1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:41:39.295441 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.295370 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ba6d4de-14e9-4bb9-b896-21b693858de1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8ba6d4de-14e9-4bb9-b896-21b693858de1" (UID: "8ba6d4de-14e9-4bb9-b896-21b693858de1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:41:39.295520 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.295492 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8ba6d4de-14e9-4bb9-b896-21b693858de1" (UID: "8ba6d4de-14e9-4bb9-b896-21b693858de1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:41:39.295520 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.295504 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8ba6d4de-14e9-4bb9-b896-21b693858de1" (UID: "8ba6d4de-14e9-4bb9-b896-21b693858de1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:41:39.295635 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.295579 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ba6d4de-14e9-4bb9-b896-21b693858de1-istiod-ca-cert\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:39.295635 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.295601 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-credential-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:39.296027 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.296002 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-data" (OuterVolumeSpecName: "istio-data") pod "8ba6d4de-14e9-4bb9-b896-21b693858de1" (UID: "8ba6d4de-14e9-4bb9-b896-21b693858de1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:41:39.297223 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.297199 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-token" (OuterVolumeSpecName: "istio-token") pod "8ba6d4de-14e9-4bb9-b896-21b693858de1" (UID: "8ba6d4de-14e9-4bb9-b896-21b693858de1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:41:39.297387 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.297369 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8ba6d4de-14e9-4bb9-b896-21b693858de1" (UID: "8ba6d4de-14e9-4bb9-b896-21b693858de1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:41:39.297538 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.297517 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8ba6d4de-14e9-4bb9-b896-21b693858de1" (UID: "8ba6d4de-14e9-4bb9-b896-21b693858de1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:41:39.297671 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.297520 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ba6d4de-14e9-4bb9-b896-21b693858de1-kube-api-access-jbtss" (OuterVolumeSpecName: "kube-api-access-jbtss") pod "8ba6d4de-14e9-4bb9-b896-21b693858de1" (UID: "8ba6d4de-14e9-4bb9-b896-21b693858de1"). InnerVolumeSpecName "kube-api-access-jbtss". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:41:39.396806 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.396779 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-workload-certs\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:39.396806 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.396802 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-podinfo\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:39.396806 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.396811 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbtss\" (UniqueName: \"kubernetes.io/projected/8ba6d4de-14e9-4bb9-b896-21b693858de1-kube-api-access-jbtss\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:39.397025 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.396822 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-data\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:39.397025 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.396833 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-token\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:39.397025 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.396844 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-workload-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:39.397025 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.396855 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ba6d4de-14e9-4bb9-b896-21b693858de1-istio-envoy\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:39.567078 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.567050 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9"] Jul 02 19:41:39.570414 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.570375 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-09f1f6-openshift-default-86d868475c-h2qq9"] Jul 02 19:41:39.727795 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:39.727763 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ba6d4de-14e9-4bb9-b896-21b693858de1" path="/var/lib/kubelet/pods/8ba6d4de-14e9-4bb9-b896-21b693858de1/volumes" Jul 02 19:41:41.046190 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:41.046158 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc"] Jul 02 19:41:41.046770 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:41.046432 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" podUID="fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" containerName="istio-proxy" containerID="cri-o://0359658e7af98147ea875c14878f726eea68b451403859d42f3c41f942dbd33c" gracePeriod=30 Jul 02 19:41:42.019904 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.019876 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq"] Jul 02 19:41:42.020242 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.020229 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ba6d4de-14e9-4bb9-b896-21b693858de1" containerName="istio-proxy" Jul 02 19:41:42.020283 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.020244 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ba6d4de-14e9-4bb9-b896-21b693858de1" containerName="istio-proxy" Jul 02 19:41:42.020283 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.020262 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ba6d4de-14e9-4bb9-b896-21b693858de1" containerName="istio-proxy" Jul 02 19:41:42.020283 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.020268 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ba6d4de-14e9-4bb9-b896-21b693858de1" containerName="istio-proxy" Jul 02 19:41:42.020380 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.020325 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ba6d4de-14e9-4bb9-b896-21b693858de1" containerName="istio-proxy" Jul 02 19:41:42.020380 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.020334 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ba6d4de-14e9-4bb9-b896-21b693858de1" containerName="istio-proxy" Jul 02 19:41:42.023444 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.023423 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.025897 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.025883 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a91b31-openshift-default-dockercfg-lbbbn\"" Jul 02 19:41:42.035359 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.035336 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq"] Jul 02 19:41:42.121779 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.121741 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-envoy\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.122271 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.121787 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8138462f-87cd-440e-a73a-7dd6bdf5622a-istiod-ca-cert\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.122271 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.121880 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-data\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.122271 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.121931 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-workload-socket\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.122271 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.121987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzjdt\" (UniqueName: \"kubernetes.io/projected/8138462f-87cd-440e-a73a-7dd6bdf5622a-kube-api-access-wzjdt\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.122271 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.122055 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-workload-certs\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.122271 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.122156 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-credential-socket\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.122271 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.122194 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-podinfo\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.122271 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.122228 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-token\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.223616 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.223581 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzjdt\" (UniqueName: \"kubernetes.io/projected/8138462f-87cd-440e-a73a-7dd6bdf5622a-kube-api-access-wzjdt\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.223616 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.223620 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-workload-certs\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.223841 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.223648 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-credential-socket\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.223841 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.223761 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-podinfo\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.223841 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.223799 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-token\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.224030 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.223886 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-envoy\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.224030 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.223910 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8138462f-87cd-440e-a73a-7dd6bdf5622a-istiod-ca-cert\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.224030 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.223953 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-data\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.224030 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.223992 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-workload-socket\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.224030 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.223992 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-workload-certs\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.224295 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.224029 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-credential-socket\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.224333 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.224309 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-data\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.224474 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.224453 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-workload-socket\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.224643 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.224627 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8138462f-87cd-440e-a73a-7dd6bdf5622a-istiod-ca-cert\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.226075 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.226053 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-envoy\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.226299 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.226277 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-podinfo\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.232793 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.232766 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-token\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.232909 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.232893 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzjdt\" (UniqueName: \"kubernetes.io/projected/8138462f-87cd-440e-a73a-7dd6bdf5622a-kube-api-access-wzjdt\") pod \"e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.334194 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.334158 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:42.479475 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.479452 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq"] Jul 02 19:41:42.481385 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:41:42.481357 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8138462f_87cd_440e_a73a_7dd6bdf5622a.slice/crio-db8cb5a7136d1a6d21266bfefd0183e6c52b933cec7068f968a67f15ea9e1541 WatchSource:0}: Error finding container db8cb5a7136d1a6d21266bfefd0183e6c52b933cec7068f968a67f15ea9e1541: Status 404 returned error can't find the container with id db8cb5a7136d1a6d21266bfefd0183e6c52b933cec7068f968a67f15ea9e1541 Jul 02 19:41:42.483581 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.483546 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:41:42.483714 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.483626 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:41:42.483714 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:42.483671 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:41:43.268276 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:43.268238 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" event={"ID":"8138462f-87cd-440e-a73a-7dd6bdf5622a","Type":"ContainerStarted","Data":"96ed048177d0e0af79cd27f91cc3b82cf0471fdc880492eccb8f2c6bc007574b"} Jul 02 19:41:43.268689 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:43.268291 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" event={"ID":"8138462f-87cd-440e-a73a-7dd6bdf5622a","Type":"ContainerStarted","Data":"db8cb5a7136d1a6d21266bfefd0183e6c52b933cec7068f968a67f15ea9e1541"} Jul 02 19:41:43.297794 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:43.297749 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" podStartSLOduration=2.297733964 podStartE2EDuration="2.297733964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:41:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:41:43.29595522 +0000 UTC m=+2046.098770674" watchObservedRunningTime="2026-07-02 19:41:43.297733964 +0000 UTC m=+2046.100549481" Jul 02 19:41:43.334465 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:43.334431 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:43.340109 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:43.340091 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:44.273295 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:44.273258 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:44.274169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:44.274151 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:41:46.284898 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.284868 2578 generic.go:356] "Generic (PLEG): container finished" podID="fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" containerID="0359658e7af98147ea875c14878f726eea68b451403859d42f3c41f942dbd33c" exitCode=0 Jul 02 19:41:46.285290 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.284965 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" event={"ID":"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae","Type":"ContainerDied","Data":"0359658e7af98147ea875c14878f726eea68b451403859d42f3c41f942dbd33c"} Jul 02 19:41:46.285290 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.285053 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" event={"ID":"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae","Type":"ContainerDied","Data":"947c1a77a47a0855b5290d47761e05d4481c17141a088979d63c097856106b4e"} Jul 02 19:41:46.285290 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.285079 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="947c1a77a47a0855b5290d47761e05d4481c17141a088979d63c097856106b4e" Jul 02 19:41:46.285290 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.285097 2578 scope.go:117] "RemoveContainer" containerID="240f26cb18d1188c39f159f40f1cbd06cf6a5304a4bd33988902744ecd285d3f" Jul 02 19:41:46.301169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.301151 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:41:46.359958 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.359935 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-data\") pod \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " Jul 02 19:41:46.360131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.359991 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-podinfo\") pod \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " Jul 02 19:41:46.360131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.360009 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-credential-socket\") pod \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " Jul 02 19:41:46.360131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.360040 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-token\") pod \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " Jul 02 19:41:46.360131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.360057 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istiod-ca-cert\") pod \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " Jul 02 19:41:46.360131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.360091 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-workload-socket\") pod \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " Jul 02 19:41:46.360424 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.360135 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-workload-certs\") pod \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " Jul 02 19:41:46.360424 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.360204 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5mjn\" (UniqueName: \"kubernetes.io/projected/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-kube-api-access-z5mjn\") pod \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " Jul 02 19:41:46.360424 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.360233 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-envoy\") pod \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\" (UID: \"fcdcdff0-49b1-4d33-95b2-2b66c07d5fae\") " Jul 02 19:41:46.360594 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.360419 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" (UID: "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:41:46.360594 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.360505 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" (UID: "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:41:46.360709 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.360660 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" (UID: "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:41:46.360709 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.360696 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istiod-ca-cert\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:46.361486 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.360714 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-credential-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:46.361486 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.360862 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-data" (OuterVolumeSpecName: "istio-data") pod "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" (UID: "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:41:46.361486 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.360966 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" (UID: "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:41:46.362581 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.362552 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" (UID: "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:41:46.362786 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.362765 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-token" (OuterVolumeSpecName: "istio-token") pod "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" (UID: "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:41:46.362978 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.362961 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" (UID: "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:41:46.363037 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.363015 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-kube-api-access-z5mjn" (OuterVolumeSpecName: "kube-api-access-z5mjn") pod "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" (UID: "fcdcdff0-49b1-4d33-95b2-2b66c07d5fae"). InnerVolumeSpecName "kube-api-access-z5mjn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:41:46.462080 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.462056 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5mjn\" (UniqueName: \"kubernetes.io/projected/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-kube-api-access-z5mjn\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:46.462080 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.462078 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-envoy\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:46.462222 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.462089 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-data\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:46.462222 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.462098 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-podinfo\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:46.462222 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.462105 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-istio-token\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:46.462222 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.462114 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-workload-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:46.462222 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:46.462122 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae-workload-certs\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:41:47.292704 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:47.292675 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc" Jul 02 19:41:47.341812 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:47.341780 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc"] Jul 02 19:41:47.348091 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:47.348067 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-cc922a-openshift-default-876bbf67-c5qqc"] Jul 02 19:41:47.727442 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:47.727414 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" path="/var/lib/kubelet/pods/fcdcdff0-49b1-4d33-95b2-2b66c07d5fae/volumes" Jul 02 19:41:50.850499 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.850467 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx"] Jul 02 19:41:50.851021 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.851006 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" containerName="istio-proxy" Jul 02 19:41:50.851070 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.851024 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" containerName="istio-proxy" Jul 02 19:41:50.851070 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.851050 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" containerName="istio-proxy" Jul 02 19:41:50.851070 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.851060 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" containerName="istio-proxy" Jul 02 19:41:50.851163 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.851141 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" containerName="istio-proxy" Jul 02 19:41:50.855815 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.855794 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.858190 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.858172 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-5a2a19-openshift-default-dockercfg-qmkt5\"" Jul 02 19:41:50.864820 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.864796 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx"] Jul 02 19:41:50.896794 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.896769 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-workload-socket\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.896942 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.896799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-podinfo\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.896942 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.896878 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvltm\" (UniqueName: \"kubernetes.io/projected/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-kube-api-access-bvltm\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.896942 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.896937 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-credential-socket\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.897069 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.896965 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-data\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.897069 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.896979 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-token\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.897069 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.897000 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-workload-certs\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.897069 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.897014 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istiod-ca-cert\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.897069 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.897068 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-envoy\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.998110 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.998070 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-envoy\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.998313 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.998134 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-workload-socket\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.998313 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.998157 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-podinfo\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.998313 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.998218 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvltm\" (UniqueName: \"kubernetes.io/projected/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-kube-api-access-bvltm\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.998313 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.998266 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-credential-socket\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.998586 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.998328 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-data\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.998586 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.998377 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-token\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.998586 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.998430 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-workload-certs\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.998586 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.998455 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istiod-ca-cert\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.998823 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.998663 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-workload-socket\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.998823 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.998707 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-credential-socket\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.998960 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.998931 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-data\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.998960 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.998941 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-workload-certs\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:50.999318 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:50.999297 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istiod-ca-cert\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:51.000685 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:51.000658 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-envoy\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:51.000873 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:51.000857 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-podinfo\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:51.007471 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:51.007452 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-token\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:51.007575 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:51.007556 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvltm\" (UniqueName: \"kubernetes.io/projected/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-kube-api-access-bvltm\") pod \"e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:51.166787 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:51.166708 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:51.314682 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:51.314662 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx"] Jul 02 19:41:51.316723 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:41:51.316696 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dbe0a14_c6e1_499b_aa42_08fc9ee15f70.slice/crio-7b1e6d981ff226b596ffa50103a287534c8e5f2d635c22325ff72ad61fb65721 WatchSource:0}: Error finding container 7b1e6d981ff226b596ffa50103a287534c8e5f2d635c22325ff72ad61fb65721: Status 404 returned error can't find the container with id 7b1e6d981ff226b596ffa50103a287534c8e5f2d635c22325ff72ad61fb65721 Jul 02 19:41:51.319167 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:51.319133 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:41:51.319293 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:51.319205 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:41:51.319293 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:51.319250 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:41:52.321354 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:52.321320 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" event={"ID":"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70","Type":"ContainerStarted","Data":"073434dea091307dbf2fe53a603a6f06fe060b206af05fb3631fc348b895acf5"} Jul 02 19:41:52.321767 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:52.321375 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" event={"ID":"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70","Type":"ContainerStarted","Data":"7b1e6d981ff226b596ffa50103a287534c8e5f2d635c22325ff72ad61fb65721"} Jul 02 19:41:52.343843 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:52.343771 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" podStartSLOduration=2.3437494389999998 podStartE2EDuration="2.343749439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:41:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:41:52.340480773 +0000 UTC m=+2055.143296264" watchObservedRunningTime="2026-07-02 19:41:52.343749439 +0000 UTC m=+2055.146564891" Jul 02 19:41:53.166989 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:53.166952 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:53.171954 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:53.171931 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:53.326530 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:53.326506 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:53.327357 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:53.327340 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:41:59.275372 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:59.275334 2578 patch_prober.go:28] interesting pod/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 02 19:41:59.275804 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:59.275387 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" podUID="8138462f-87cd-440e-a73a-7dd6bdf5622a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 02 19:41:59.367759 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:59.367725 2578 generic.go:356] "Generic (PLEG): container finished" podID="8138462f-87cd-440e-a73a-7dd6bdf5622a" containerID="96ed048177d0e0af79cd27f91cc3b82cf0471fdc880492eccb8f2c6bc007574b" exitCode=0 Jul 02 19:41:59.367932 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:59.367801 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" event={"ID":"8138462f-87cd-440e-a73a-7dd6bdf5622a","Type":"ContainerDied","Data":"96ed048177d0e0af79cd27f91cc3b82cf0471fdc880492eccb8f2c6bc007574b"} Jul 02 19:41:59.368219 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:59.368204 2578 scope.go:117] "RemoveContainer" containerID="96ed048177d0e0af79cd27f91cc3b82cf0471fdc880492eccb8f2c6bc007574b" Jul 02 19:41:59.369215 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:59.369185 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:41:59.369323 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:59.369307 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:41:59.369387 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:41:59.369361 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:00.379883 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:00.379782 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" event={"ID":"8138462f-87cd-440e-a73a-7dd6bdf5622a","Type":"ContainerStarted","Data":"261fca9b9767ac2234b19b917b85a09dba46df4654b0a07e4129dd865e41a7d0"} Jul 02 19:42:01.334786 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:01.334745 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:42:01.335154 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:01.335121 2578 patch_prober.go:28] interesting pod/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 02 19:42:01.335232 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:01.335189 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" podUID="8138462f-87cd-440e-a73a-7dd6bdf5622a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 02 19:42:02.334842 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:02.334804 2578 patch_prober.go:28] interesting pod/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 02 19:42:02.335440 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:02.335388 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" podUID="8138462f-87cd-440e-a73a-7dd6bdf5622a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 02 19:42:03.335652 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:03.335611 2578 patch_prober.go:28] interesting pod/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 02 19:42:03.336131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:03.335677 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" podUID="8138462f-87cd-440e-a73a-7dd6bdf5622a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 02 19:42:04.338983 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:04.338948 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:42:04.339353 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:04.339331 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:42:04.340082 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:04.340061 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:42:06.420924 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:06.420890 2578 generic.go:356] "Generic (PLEG): container finished" podID="7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" containerID="073434dea091307dbf2fe53a603a6f06fe060b206af05fb3631fc348b895acf5" exitCode=0 Jul 02 19:42:06.421494 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:06.420964 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" event={"ID":"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70","Type":"ContainerDied","Data":"073434dea091307dbf2fe53a603a6f06fe060b206af05fb3631fc348b895acf5"} Jul 02 19:42:06.421567 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:06.421507 2578 scope.go:117] "RemoveContainer" containerID="073434dea091307dbf2fe53a603a6f06fe060b206af05fb3631fc348b895acf5" Jul 02 19:42:06.422667 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:06.422620 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:06.422759 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:06.422715 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:06.422799 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:06.422769 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:07.428780 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:07.428745 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" event={"ID":"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70","Type":"ContainerStarted","Data":"b84d4e1b62094e6f15358deb2d5077462f7a514f1bb24086affbb3b986cc3018"} Jul 02 19:42:08.167837 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:08.167804 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:42:08.168840 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:08.168812 2578 patch_prober.go:28] interesting pod/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 02 19:42:08.168973 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:08.168860 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" podUID="7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 02 19:42:08.327306 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:08.327274 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:42:09.167630 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:09.167595 2578 patch_prober.go:28] interesting pod/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 02 19:42:09.168063 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:09.167649 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" podUID="7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 02 19:42:10.167350 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:10.167315 2578 patch_prober.go:28] interesting pod/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 02 19:42:10.167543 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:10.167380 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" podUID="7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 02 19:42:11.168146 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:11.168111 2578 patch_prober.go:28] interesting pod/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 02 19:42:11.168606 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:11.168164 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" podUID="7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 02 19:42:12.171424 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:12.171378 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:42:12.172375 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:12.172360 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:42:13.639291 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:13.639260 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq"] Jul 02 19:42:13.639766 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:13.639507 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" podUID="8138462f-87cd-440e-a73a-7dd6bdf5622a" containerName="istio-proxy" containerID="cri-o://261fca9b9767ac2234b19b917b85a09dba46df4654b0a07e4129dd865e41a7d0" gracePeriod=30 Jul 02 19:42:18.901565 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.901540 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:42:18.971539 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.971467 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzjdt\" (UniqueName: \"kubernetes.io/projected/8138462f-87cd-440e-a73a-7dd6bdf5622a-kube-api-access-wzjdt\") pod \"8138462f-87cd-440e-a73a-7dd6bdf5622a\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " Jul 02 19:42:18.971539 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.971511 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-podinfo\") pod \"8138462f-87cd-440e-a73a-7dd6bdf5622a\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " Jul 02 19:42:18.971724 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.971557 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-envoy\") pod \"8138462f-87cd-440e-a73a-7dd6bdf5622a\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " Jul 02 19:42:18.971724 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.971595 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-credential-socket\") pod \"8138462f-87cd-440e-a73a-7dd6bdf5622a\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " Jul 02 19:42:18.971724 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.971611 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-workload-certs\") pod \"8138462f-87cd-440e-a73a-7dd6bdf5622a\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " Jul 02 19:42:18.971724 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.971631 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-token\") pod \"8138462f-87cd-440e-a73a-7dd6bdf5622a\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " Jul 02 19:42:18.971724 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.971680 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-data\") pod \"8138462f-87cd-440e-a73a-7dd6bdf5622a\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " Jul 02 19:42:18.971724 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.971707 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-workload-socket\") pod \"8138462f-87cd-440e-a73a-7dd6bdf5622a\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " Jul 02 19:42:18.972023 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.971744 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8138462f-87cd-440e-a73a-7dd6bdf5622a-istiod-ca-cert\") pod \"8138462f-87cd-440e-a73a-7dd6bdf5622a\" (UID: \"8138462f-87cd-440e-a73a-7dd6bdf5622a\") " Jul 02 19:42:18.972023 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.971923 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8138462f-87cd-440e-a73a-7dd6bdf5622a" (UID: "8138462f-87cd-440e-a73a-7dd6bdf5622a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:42:18.972023 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.971945 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8138462f-87cd-440e-a73a-7dd6bdf5622a" (UID: "8138462f-87cd-440e-a73a-7dd6bdf5622a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:42:18.972178 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.972102 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-credential-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:18.972178 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.972107 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8138462f-87cd-440e-a73a-7dd6bdf5622a" (UID: "8138462f-87cd-440e-a73a-7dd6bdf5622a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:42:18.972178 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.972122 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-workload-certs\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:18.972455 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.972378 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8138462f-87cd-440e-a73a-7dd6bdf5622a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8138462f-87cd-440e-a73a-7dd6bdf5622a" (UID: "8138462f-87cd-440e-a73a-7dd6bdf5622a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:42:18.972808 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.972759 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-data" (OuterVolumeSpecName: "istio-data") pod "8138462f-87cd-440e-a73a-7dd6bdf5622a" (UID: "8138462f-87cd-440e-a73a-7dd6bdf5622a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:42:18.974186 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.974158 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8138462f-87cd-440e-a73a-7dd6bdf5622a-kube-api-access-wzjdt" (OuterVolumeSpecName: "kube-api-access-wzjdt") pod "8138462f-87cd-440e-a73a-7dd6bdf5622a" (UID: "8138462f-87cd-440e-a73a-7dd6bdf5622a"). InnerVolumeSpecName "kube-api-access-wzjdt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:42:18.974565 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.974534 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8138462f-87cd-440e-a73a-7dd6bdf5622a" (UID: "8138462f-87cd-440e-a73a-7dd6bdf5622a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:42:18.974687 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.974607 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8138462f-87cd-440e-a73a-7dd6bdf5622a" (UID: "8138462f-87cd-440e-a73a-7dd6bdf5622a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:42:18.975177 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:18.975148 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-token" (OuterVolumeSpecName: "istio-token") pod "8138462f-87cd-440e-a73a-7dd6bdf5622a" (UID: "8138462f-87cd-440e-a73a-7dd6bdf5622a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:42:19.072681 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.072655 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-data\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:19.072681 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.072679 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-workload-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:19.072843 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.072688 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8138462f-87cd-440e-a73a-7dd6bdf5622a-istiod-ca-cert\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:19.072843 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.072697 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzjdt\" (UniqueName: \"kubernetes.io/projected/8138462f-87cd-440e-a73a-7dd6bdf5622a-kube-api-access-wzjdt\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:19.072843 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.072707 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-podinfo\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:19.072843 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.072715 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-envoy\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:19.072843 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.072725 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8138462f-87cd-440e-a73a-7dd6bdf5622a-istio-token\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:19.505770 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.505737 2578 generic.go:356] "Generic (PLEG): container finished" podID="8138462f-87cd-440e-a73a-7dd6bdf5622a" containerID="261fca9b9767ac2234b19b917b85a09dba46df4654b0a07e4129dd865e41a7d0" exitCode=0 Jul 02 19:42:19.505942 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.505780 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" Jul 02 19:42:19.505942 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.505812 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" event={"ID":"8138462f-87cd-440e-a73a-7dd6bdf5622a","Type":"ContainerDied","Data":"261fca9b9767ac2234b19b917b85a09dba46df4654b0a07e4129dd865e41a7d0"} Jul 02 19:42:19.505942 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.505866 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq" event={"ID":"8138462f-87cd-440e-a73a-7dd6bdf5622a","Type":"ContainerDied","Data":"db8cb5a7136d1a6d21266bfefd0183e6c52b933cec7068f968a67f15ea9e1541"} Jul 02 19:42:19.505942 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.505888 2578 scope.go:117] "RemoveContainer" containerID="261fca9b9767ac2234b19b917b85a09dba46df4654b0a07e4129dd865e41a7d0" Jul 02 19:42:19.515671 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.515655 2578 scope.go:117] "RemoveContainer" containerID="96ed048177d0e0af79cd27f91cc3b82cf0471fdc880492eccb8f2c6bc007574b" Jul 02 19:42:19.523758 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.523742 2578 scope.go:117] "RemoveContainer" containerID="261fca9b9767ac2234b19b917b85a09dba46df4654b0a07e4129dd865e41a7d0" Jul 02 19:42:19.524028 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:42:19.524011 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"261fca9b9767ac2234b19b917b85a09dba46df4654b0a07e4129dd865e41a7d0\": container with ID starting with 261fca9b9767ac2234b19b917b85a09dba46df4654b0a07e4129dd865e41a7d0 not found: ID does not exist" containerID="261fca9b9767ac2234b19b917b85a09dba46df4654b0a07e4129dd865e41a7d0" Jul 02 19:42:19.524075 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.524035 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"261fca9b9767ac2234b19b917b85a09dba46df4654b0a07e4129dd865e41a7d0"} err="failed to get container status \"261fca9b9767ac2234b19b917b85a09dba46df4654b0a07e4129dd865e41a7d0\": rpc error: code = NotFound desc = could not find container \"261fca9b9767ac2234b19b917b85a09dba46df4654b0a07e4129dd865e41a7d0\": container with ID starting with 261fca9b9767ac2234b19b917b85a09dba46df4654b0a07e4129dd865e41a7d0 not found: ID does not exist" Jul 02 19:42:19.524075 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.524052 2578 scope.go:117] "RemoveContainer" containerID="96ed048177d0e0af79cd27f91cc3b82cf0471fdc880492eccb8f2c6bc007574b" Jul 02 19:42:19.524260 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:42:19.524244 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96ed048177d0e0af79cd27f91cc3b82cf0471fdc880492eccb8f2c6bc007574b\": container with ID starting with 96ed048177d0e0af79cd27f91cc3b82cf0471fdc880492eccb8f2c6bc007574b not found: ID does not exist" containerID="96ed048177d0e0af79cd27f91cc3b82cf0471fdc880492eccb8f2c6bc007574b" Jul 02 19:42:19.524300 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.524265 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96ed048177d0e0af79cd27f91cc3b82cf0471fdc880492eccb8f2c6bc007574b"} err="failed to get container status \"96ed048177d0e0af79cd27f91cc3b82cf0471fdc880492eccb8f2c6bc007574b\": rpc error: code = NotFound desc = could not find container \"96ed048177d0e0af79cd27f91cc3b82cf0471fdc880492eccb8f2c6bc007574b\": container with ID starting with 96ed048177d0e0af79cd27f91cc3b82cf0471fdc880492eccb8f2c6bc007574b not found: ID does not exist" Jul 02 19:42:19.536042 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.536020 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq"] Jul 02 19:42:19.542804 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.542784 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a91b31-openshift-default-59f7cd8bdf-54ghq"] Jul 02 19:42:19.727809 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:19.727780 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8138462f-87cd-440e-a73a-7dd6bdf5622a" path="/var/lib/kubelet/pods/8138462f-87cd-440e-a73a-7dd6bdf5622a/volumes" Jul 02 19:42:20.754496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:20.754469 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx"] Jul 02 19:42:20.754921 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:20.754649 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" podUID="7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" containerName="istio-proxy" containerID="cri-o://b84d4e1b62094e6f15358deb2d5077462f7a514f1bb24086affbb3b986cc3018" gracePeriod=30 Jul 02 19:42:21.654943 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.654914 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn"] Jul 02 19:42:21.655286 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.655275 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8138462f-87cd-440e-a73a-7dd6bdf5622a" containerName="istio-proxy" Jul 02 19:42:21.655334 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.655289 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8138462f-87cd-440e-a73a-7dd6bdf5622a" containerName="istio-proxy" Jul 02 19:42:21.655334 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.655298 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8138462f-87cd-440e-a73a-7dd6bdf5622a" containerName="istio-proxy" Jul 02 19:42:21.655334 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.655304 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8138462f-87cd-440e-a73a-7dd6bdf5622a" containerName="istio-proxy" Jul 02 19:42:21.655450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.655366 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8138462f-87cd-440e-a73a-7dd6bdf5622a" containerName="istio-proxy" Jul 02 19:42:21.655450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.655375 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcdcdff0-49b1-4d33-95b2-2b66c07d5fae" containerName="istio-proxy" Jul 02 19:42:21.655450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.655382 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8138462f-87cd-440e-a73a-7dd6bdf5622a" containerName="istio-proxy" Jul 02 19:42:21.659880 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.659862 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.662352 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.662333 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-67cf22-openshift-default-dockercfg-ptnhx\"" Jul 02 19:42:21.671020 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.670996 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn"] Jul 02 19:42:21.692061 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.692035 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-credential-socket\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.692169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.692073 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-workload-certs\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.692169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.692101 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-data\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.692169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.692160 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bac33b1-cbcb-4d80-a665-749c69797d04-istiod-ca-cert\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.692294 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.692199 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-podinfo\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.692294 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.692237 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-workload-socket\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.692294 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.692269 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg5ts\" (UniqueName: \"kubernetes.io/projected/6bac33b1-cbcb-4d80-a665-749c69797d04-kube-api-access-fg5ts\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.692452 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.692307 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-token\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.692452 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.692331 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-envoy\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.793063 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.793023 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bac33b1-cbcb-4d80-a665-749c69797d04-istiod-ca-cert\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.793581 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.793081 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-podinfo\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.793581 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.793132 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-workload-socket\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.793581 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.793309 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg5ts\" (UniqueName: \"kubernetes.io/projected/6bac33b1-cbcb-4d80-a665-749c69797d04-kube-api-access-fg5ts\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.793581 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.793367 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-token\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.793581 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.793448 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-envoy\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.793581 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.793497 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-credential-socket\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.793581 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.793526 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-workload-certs\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.793581 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.793557 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-data\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.794039 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.793698 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-workload-socket\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.794039 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.793917 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-data\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.794039 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.793947 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bac33b1-cbcb-4d80-a665-749c69797d04-istiod-ca-cert\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.794233 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.794206 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-workload-certs\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.794311 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.794290 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-credential-socket\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.795998 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.795969 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-envoy\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.796112 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.796089 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-podinfo\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.802224 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.802199 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-token\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.802314 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.802249 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg5ts\" (UniqueName: \"kubernetes.io/projected/6bac33b1-cbcb-4d80-a665-749c69797d04-kube-api-access-fg5ts\") pod \"e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:21.972981 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:21.972902 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:22.112786 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:22.112762 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn"] Jul 02 19:42:22.118149 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:42:22.118118 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bac33b1_cbcb_4d80_a665_749c69797d04.slice/crio-67070a87ab926586a805018f49720be31d681674f67c4a093409ddbc01a7533a WatchSource:0}: Error finding container 67070a87ab926586a805018f49720be31d681674f67c4a093409ddbc01a7533a: Status 404 returned error can't find the container with id 67070a87ab926586a805018f49720be31d681674f67c4a093409ddbc01a7533a Jul 02 19:42:22.120132 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:22.120100 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:22.120210 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:22.120170 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:22.120210 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:22.120198 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:22.526092 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:22.526059 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" event={"ID":"6bac33b1-cbcb-4d80-a665-749c69797d04","Type":"ContainerStarted","Data":"25395cd58d328f2872e3c7603a684ecbae760b0247fe7996ffcf6147d5b2b11e"} Jul 02 19:42:22.526294 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:22.526120 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" event={"ID":"6bac33b1-cbcb-4d80-a665-749c69797d04","Type":"ContainerStarted","Data":"67070a87ab926586a805018f49720be31d681674f67c4a093409ddbc01a7533a"} Jul 02 19:42:22.548669 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:22.548613 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" podStartSLOduration=1.548592497 podStartE2EDuration="1.548592497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:42:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:42:22.547013306 +0000 UTC m=+2085.349828757" watchObservedRunningTime="2026-07-02 19:42:22.548592497 +0000 UTC m=+2085.351407950" Jul 02 19:42:22.973488 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:22.973457 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:23.978931 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:23.978902 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:24.538253 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:24.538226 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:24.539426 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:24.539373 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:26.010972 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.010949 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:42:26.036376 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.033167 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-workload-socket\") pod \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " Jul 02 19:42:26.036376 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.033302 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-token\") pod \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " Jul 02 19:42:26.036376 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.033358 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-data\") pod \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " Jul 02 19:42:26.036376 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.033725 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-workload-certs\") pod \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " Jul 02 19:42:26.036376 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.035431 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvltm\" (UniqueName: \"kubernetes.io/projected/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-kube-api-access-bvltm\") pod \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " Jul 02 19:42:26.036376 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.035484 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istiod-ca-cert\") pod \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " Jul 02 19:42:26.036376 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.035546 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-credential-socket\") pod \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " Jul 02 19:42:26.036376 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.035589 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-envoy\") pod \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " Jul 02 19:42:26.036376 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.035618 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-podinfo\") pod \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\" (UID: \"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70\") " Jul 02 19:42:26.040586 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.040539 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" (UID: "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:42:26.040708 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.040681 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" (UID: "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:42:26.040784 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.040632 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" (UID: "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:42:26.040971 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.040947 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" (UID: "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:42:26.041158 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.041118 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-data" (OuterVolumeSpecName: "istio-data") pod "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" (UID: "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:42:26.044008 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.043292 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-token" (OuterVolumeSpecName: "istio-token") pod "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" (UID: "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:42:26.045116 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.045079 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" (UID: "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:42:26.045217 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.045199 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" (UID: "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:42:26.045434 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.045413 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-kube-api-access-bvltm" (OuterVolumeSpecName: "kube-api-access-bvltm") pod "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" (UID: "7dbe0a14-c6e1-499b-aa42-08fc9ee15f70"). InnerVolumeSpecName "kube-api-access-bvltm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:42:26.136959 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.136930 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-envoy\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:26.136959 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.136958 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-podinfo\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:26.137135 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.136968 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-workload-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:26.137135 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.136977 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-token\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:26.137135 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.136985 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istio-data\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:26.137135 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.136992 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-workload-certs\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:26.137135 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.137018 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvltm\" (UniqueName: \"kubernetes.io/projected/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-kube-api-access-bvltm\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:26.137135 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.137029 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-istiod-ca-cert\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:26.137135 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.137038 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70-credential-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:42:26.551593 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.551562 2578 generic.go:356] "Generic (PLEG): container finished" podID="7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" containerID="b84d4e1b62094e6f15358deb2d5077462f7a514f1bb24086affbb3b986cc3018" exitCode=0 Jul 02 19:42:26.551785 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.551621 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" Jul 02 19:42:26.551785 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.551652 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" event={"ID":"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70","Type":"ContainerDied","Data":"b84d4e1b62094e6f15358deb2d5077462f7a514f1bb24086affbb3b986cc3018"} Jul 02 19:42:26.551785 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.551742 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx" event={"ID":"7dbe0a14-c6e1-499b-aa42-08fc9ee15f70","Type":"ContainerDied","Data":"7b1e6d981ff226b596ffa50103a287534c8e5f2d635c22325ff72ad61fb65721"} Jul 02 19:42:26.551785 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.551772 2578 scope.go:117] "RemoveContainer" containerID="b84d4e1b62094e6f15358deb2d5077462f7a514f1bb24086affbb3b986cc3018" Jul 02 19:42:26.561523 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.561508 2578 scope.go:117] "RemoveContainer" containerID="073434dea091307dbf2fe53a603a6f06fe060b206af05fb3631fc348b895acf5" Jul 02 19:42:26.569641 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.569623 2578 scope.go:117] "RemoveContainer" containerID="b84d4e1b62094e6f15358deb2d5077462f7a514f1bb24086affbb3b986cc3018" Jul 02 19:42:26.569875 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:42:26.569857 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b84d4e1b62094e6f15358deb2d5077462f7a514f1bb24086affbb3b986cc3018\": container with ID starting with b84d4e1b62094e6f15358deb2d5077462f7a514f1bb24086affbb3b986cc3018 not found: ID does not exist" containerID="b84d4e1b62094e6f15358deb2d5077462f7a514f1bb24086affbb3b986cc3018" Jul 02 19:42:26.569911 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.569884 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b84d4e1b62094e6f15358deb2d5077462f7a514f1bb24086affbb3b986cc3018"} err="failed to get container status \"b84d4e1b62094e6f15358deb2d5077462f7a514f1bb24086affbb3b986cc3018\": rpc error: code = NotFound desc = could not find container \"b84d4e1b62094e6f15358deb2d5077462f7a514f1bb24086affbb3b986cc3018\": container with ID starting with b84d4e1b62094e6f15358deb2d5077462f7a514f1bb24086affbb3b986cc3018 not found: ID does not exist" Jul 02 19:42:26.569911 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.569901 2578 scope.go:117] "RemoveContainer" containerID="073434dea091307dbf2fe53a603a6f06fe060b206af05fb3631fc348b895acf5" Jul 02 19:42:26.570120 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:42:26.570106 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"073434dea091307dbf2fe53a603a6f06fe060b206af05fb3631fc348b895acf5\": container with ID starting with 073434dea091307dbf2fe53a603a6f06fe060b206af05fb3631fc348b895acf5 not found: ID does not exist" containerID="073434dea091307dbf2fe53a603a6f06fe060b206af05fb3631fc348b895acf5" Jul 02 19:42:26.570154 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.570124 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"073434dea091307dbf2fe53a603a6f06fe060b206af05fb3631fc348b895acf5"} err="failed to get container status \"073434dea091307dbf2fe53a603a6f06fe060b206af05fb3631fc348b895acf5\": rpc error: code = NotFound desc = could not find container \"073434dea091307dbf2fe53a603a6f06fe060b206af05fb3631fc348b895acf5\": container with ID starting with 073434dea091307dbf2fe53a603a6f06fe060b206af05fb3631fc348b895acf5 not found: ID does not exist" Jul 02 19:42:26.574716 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.574693 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx"] Jul 02 19:42:26.578316 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:26.578296 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-5a2a19-openshift-default-c5f95c65f-kwmhx"] Jul 02 19:42:27.728046 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:27.728010 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" path="/var/lib/kubelet/pods/7dbe0a14-c6e1-499b-aa42-08fc9ee15f70/volumes" Jul 02 19:42:29.840855 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.840822 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4"] Jul 02 19:42:29.841246 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.841212 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" containerName="istio-proxy" Jul 02 19:42:29.841246 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.841225 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" containerName="istio-proxy" Jul 02 19:42:29.841246 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.841244 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" containerName="istio-proxy" Jul 02 19:42:29.841354 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.841250 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" containerName="istio-proxy" Jul 02 19:42:29.841354 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.841315 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" containerName="istio-proxy" Jul 02 19:42:29.841354 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.841325 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dbe0a14-c6e1-499b-aa42-08fc9ee15f70" containerName="istio-proxy" Jul 02 19:42:29.845928 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.845908 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:29.851815 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.851795 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-0c31dc-openshift-default-dockercfg-wbt5c\"" Jul 02 19:42:29.871503 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.871482 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4"] Jul 02 19:42:29.969800 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.969771 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-credential-socket\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:29.969978 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.969807 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw9lb\" (UniqueName: \"kubernetes.io/projected/c95d159a-e242-4468-9a93-8068a46a5576-kube-api-access-cw9lb\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:29.969978 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.969841 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c95d159a-e242-4468-9a93-8068a46a5576-istiod-ca-cert\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:29.969978 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.969920 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-workload-certs\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:29.969978 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.969956 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c95d159a-e242-4468-9a93-8068a46a5576-istio-podinfo\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:29.970126 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.970008 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-workload-socket\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:29.970126 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.970049 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-istio-envoy\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:29.970126 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.970065 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c95d159a-e242-4468-9a93-8068a46a5576-istio-token\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:29.970126 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:29.970083 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-istio-data\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.070790 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.070752 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c95d159a-e242-4468-9a93-8068a46a5576-istiod-ca-cert\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.070996 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.070804 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-workload-certs\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.070996 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.070894 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c95d159a-e242-4468-9a93-8068a46a5576-istio-podinfo\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.070996 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.070941 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-workload-socket\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.070996 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.070975 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-istio-envoy\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.070996 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.070998 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c95d159a-e242-4468-9a93-8068a46a5576-istio-token\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.071287 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.071022 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-istio-data\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.071287 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.071051 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-credential-socket\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.071287 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.071079 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw9lb\" (UniqueName: \"kubernetes.io/projected/c95d159a-e242-4468-9a93-8068a46a5576-kube-api-access-cw9lb\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.071488 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.071285 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-workload-certs\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.071488 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.071468 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-istio-data\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.071602 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.071467 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-workload-socket\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.071602 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.071574 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-credential-socket\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.071684 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.071597 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c95d159a-e242-4468-9a93-8068a46a5576-istiod-ca-cert\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.073383 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.073354 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-istio-envoy\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.073671 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.073653 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c95d159a-e242-4468-9a93-8068a46a5576-istio-podinfo\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.079722 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.079698 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c95d159a-e242-4468-9a93-8068a46a5576-istio-token\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.079936 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.079914 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw9lb\" (UniqueName: \"kubernetes.io/projected/c95d159a-e242-4468-9a93-8068a46a5576-kube-api-access-cw9lb\") pod \"e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.156159 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.156078 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:30.319492 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.317512 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4"] Jul 02 19:42:30.319807 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.319773 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:30.319893 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.319838 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:30.319893 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.319866 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:30.582462 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.582424 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" event={"ID":"c95d159a-e242-4468-9a93-8068a46a5576","Type":"ContainerStarted","Data":"394df70b1dd603d650efdd8fb3dc9a279fe4b4328ef855f386a2674845d8e558"} Jul 02 19:42:30.582668 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.582513 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" event={"ID":"c95d159a-e242-4468-9a93-8068a46a5576","Type":"ContainerStarted","Data":"3b55cee5e7cb7a0773c4c8dc7bb5737020cee16a295e8f6e1bfcef0d6b51db7b"} Jul 02 19:42:30.602022 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:30.601968 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" podStartSLOduration=1.601955953 podStartE2EDuration="1.601955953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:42:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:42:30.601798359 +0000 UTC m=+2093.404613829" watchObservedRunningTime="2026-07-02 19:42:30.601955953 +0000 UTC m=+2093.404771402" Jul 02 19:42:31.157066 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:31.157030 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:31.163538 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:31.163507 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:31.588172 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:31.588139 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:31.589121 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:31.589101 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:37.621883 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:37.621850 2578 generic.go:356] "Generic (PLEG): container finished" podID="6bac33b1-cbcb-4d80-a665-749c69797d04" containerID="25395cd58d328f2872e3c7603a684ecbae760b0247fe7996ffcf6147d5b2b11e" exitCode=0 Jul 02 19:42:37.622304 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:37.621907 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" event={"ID":"6bac33b1-cbcb-4d80-a665-749c69797d04","Type":"ContainerDied","Data":"25395cd58d328f2872e3c7603a684ecbae760b0247fe7996ffcf6147d5b2b11e"} Jul 02 19:42:37.622304 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:37.622287 2578 scope.go:117] "RemoveContainer" containerID="25395cd58d328f2872e3c7603a684ecbae760b0247fe7996ffcf6147d5b2b11e" Jul 02 19:42:37.623262 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:37.623229 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:37.623367 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:37.623312 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:37.623367 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:37.623361 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:38.630295 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:38.630262 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" event={"ID":"6bac33b1-cbcb-4d80-a665-749c69797d04","Type":"ContainerStarted","Data":"b29d1e8c2f50a740770730abc41c4f452a0b01cf6ad94666131ac1683ded838c"} Jul 02 19:42:38.973880 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:38.973780 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:38.974842 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:38.974814 2578 patch_prober.go:28] interesting pod/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 02 19:42:38.975069 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:38.975040 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" podUID="6bac33b1-cbcb-4d80-a665-749c69797d04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 02 19:42:39.538634 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:39.538601 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:39.973702 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:39.973655 2578 patch_prober.go:28] interesting pod/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 02 19:42:39.974225 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:39.973723 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" podUID="6bac33b1-cbcb-4d80-a665-749c69797d04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 02 19:42:40.973985 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:40.973947 2578 patch_prober.go:28] interesting pod/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 02 19:42:40.974606 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:40.974555 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" podUID="6bac33b1-cbcb-4d80-a665-749c69797d04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 02 19:42:41.973960 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:41.973919 2578 patch_prober.go:28] interesting pod/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 02 19:42:41.974164 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:41.973983 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" podUID="6bac33b1-cbcb-4d80-a665-749c69797d04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 02 19:42:42.979625 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:42.979594 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:42.980823 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:42.980777 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:42:44.674909 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:44.674823 2578 generic.go:356] "Generic (PLEG): container finished" podID="c95d159a-e242-4468-9a93-8068a46a5576" containerID="394df70b1dd603d650efdd8fb3dc9a279fe4b4328ef855f386a2674845d8e558" exitCode=0 Jul 02 19:42:44.674909 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:44.674898 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" event={"ID":"c95d159a-e242-4468-9a93-8068a46a5576","Type":"ContainerDied","Data":"394df70b1dd603d650efdd8fb3dc9a279fe4b4328ef855f386a2674845d8e558"} Jul 02 19:42:44.675326 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:44.675299 2578 scope.go:117] "RemoveContainer" containerID="394df70b1dd603d650efdd8fb3dc9a279fe4b4328ef855f386a2674845d8e558" Jul 02 19:42:44.676437 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:44.676412 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:44.676515 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:44.676475 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:44.676515 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:44.676508 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:42:45.682776 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:45.682742 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" event={"ID":"c95d159a-e242-4468-9a93-8068a46a5576","Type":"ContainerStarted","Data":"e0319fff412182d8ad80bde98116096b6f9e5b5de65db80fedf29a9ffeb248cb"} Jul 02 19:42:46.157335 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:46.157224 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:46.158739 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:46.158365 2578 patch_prober.go:28] interesting pod/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 02 19:42:46.158739 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:46.158453 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" podUID="c95d159a-e242-4468-9a93-8068a46a5576" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 02 19:42:46.588932 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:46.588903 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:47.156819 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:47.156786 2578 patch_prober.go:28] interesting pod/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 02 19:42:47.157436 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:47.157383 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" podUID="c95d159a-e242-4468-9a93-8068a46a5576" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 02 19:42:48.156777 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:48.156742 2578 patch_prober.go:28] interesting pod/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 02 19:42:48.157052 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:48.157022 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" podUID="c95d159a-e242-4468-9a93-8068a46a5576" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 02 19:42:49.156577 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:49.156536 2578 patch_prober.go:28] interesting pod/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 02 19:42:49.156799 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:49.156604 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" podUID="c95d159a-e242-4468-9a93-8068a46a5576" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 02 19:42:50.161148 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:50.161122 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:42:50.162377 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:42:50.162353 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:46:37.980750 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:46:37.980676 2578 scope.go:117] "RemoveContainer" containerID="0359658e7af98147ea875c14878f726eea68b451403859d42f3c41f942dbd33c" Jul 02 19:48:54.251167 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:54.251138 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn"] Jul 02 19:48:54.251731 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:54.251347 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" podUID="6bac33b1-cbcb-4d80-a665-749c69797d04" containerName="istio-proxy" containerID="cri-o://b29d1e8c2f50a740770730abc41c4f452a0b01cf6ad94666131ac1683ded838c" gracePeriod=30 Jul 02 19:48:57.982154 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:57.982112 2578 patch_prober.go:28] interesting pod/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 02 19:48:57.982614 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:57.982182 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" podUID="6bac33b1-cbcb-4d80-a665-749c69797d04" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 19:48:59.496113 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.496089 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:48:59.636361 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.636335 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-podinfo\") pod \"6bac33b1-cbcb-4d80-a665-749c69797d04\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " Jul 02 19:48:59.636552 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.636377 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-envoy\") pod \"6bac33b1-cbcb-4d80-a665-749c69797d04\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " Jul 02 19:48:59.636552 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.636424 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fg5ts\" (UniqueName: \"kubernetes.io/projected/6bac33b1-cbcb-4d80-a665-749c69797d04-kube-api-access-fg5ts\") pod \"6bac33b1-cbcb-4d80-a665-749c69797d04\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " Jul 02 19:48:59.636686 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.636602 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-data\") pod \"6bac33b1-cbcb-4d80-a665-749c69797d04\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " Jul 02 19:48:59.637042 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.636995 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6bac33b1-cbcb-4d80-a665-749c69797d04" (UID: "6bac33b1-cbcb-4d80-a665-749c69797d04"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:48:59.637593 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.637556 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-data" (OuterVolumeSpecName: "istio-data") pod "6bac33b1-cbcb-4d80-a665-749c69797d04" (UID: "6bac33b1-cbcb-4d80-a665-749c69797d04"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:48:59.637692 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.637613 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-credential-socket\") pod \"6bac33b1-cbcb-4d80-a665-749c69797d04\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " Jul 02 19:48:59.637692 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.637665 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-token\") pod \"6bac33b1-cbcb-4d80-a665-749c69797d04\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " Jul 02 19:48:59.638022 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.638003 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bac33b1-cbcb-4d80-a665-749c69797d04-istiod-ca-cert\") pod \"6bac33b1-cbcb-4d80-a665-749c69797d04\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " Jul 02 19:48:59.638711 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.638685 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-workload-socket\") pod \"6bac33b1-cbcb-4d80-a665-749c69797d04\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " Jul 02 19:48:59.638811 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.638734 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-workload-certs\") pod \"6bac33b1-cbcb-4d80-a665-749c69797d04\" (UID: \"6bac33b1-cbcb-4d80-a665-749c69797d04\") " Jul 02 19:48:59.638890 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.638868 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6bac33b1-cbcb-4d80-a665-749c69797d04" (UID: "6bac33b1-cbcb-4d80-a665-749c69797d04"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:48:59.639086 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.639066 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6bac33b1-cbcb-4d80-a665-749c69797d04" (UID: "6bac33b1-cbcb-4d80-a665-749c69797d04"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:48:59.639164 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.639089 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-data\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:48:59.639164 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.639109 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-credential-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:48:59.639164 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.639123 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-workload-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:48:59.639336 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.639205 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bac33b1-cbcb-4d80-a665-749c69797d04-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6bac33b1-cbcb-4d80-a665-749c69797d04" (UID: "6bac33b1-cbcb-4d80-a665-749c69797d04"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:48:59.639476 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.639452 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6bac33b1-cbcb-4d80-a665-749c69797d04" (UID: "6bac33b1-cbcb-4d80-a665-749c69797d04"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:48:59.639832 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.639815 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6bac33b1-cbcb-4d80-a665-749c69797d04" (UID: "6bac33b1-cbcb-4d80-a665-749c69797d04"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:48:59.639908 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.639859 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bac33b1-cbcb-4d80-a665-749c69797d04-kube-api-access-fg5ts" (OuterVolumeSpecName: "kube-api-access-fg5ts") pod "6bac33b1-cbcb-4d80-a665-749c69797d04" (UID: "6bac33b1-cbcb-4d80-a665-749c69797d04"). InnerVolumeSpecName "kube-api-access-fg5ts". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:48:59.639908 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.639871 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-token" (OuterVolumeSpecName: "istio-token") pod "6bac33b1-cbcb-4d80-a665-749c69797d04" (UID: "6bac33b1-cbcb-4d80-a665-749c69797d04"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:48:59.701808 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.701779 2578 generic.go:356] "Generic (PLEG): container finished" podID="6bac33b1-cbcb-4d80-a665-749c69797d04" containerID="b29d1e8c2f50a740770730abc41c4f452a0b01cf6ad94666131ac1683ded838c" exitCode=0 Jul 02 19:48:59.701921 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.701826 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" Jul 02 19:48:59.701921 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.701827 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" event={"ID":"6bac33b1-cbcb-4d80-a665-749c69797d04","Type":"ContainerDied","Data":"b29d1e8c2f50a740770730abc41c4f452a0b01cf6ad94666131ac1683ded838c"} Jul 02 19:48:59.701994 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.701939 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn" event={"ID":"6bac33b1-cbcb-4d80-a665-749c69797d04","Type":"ContainerDied","Data":"67070a87ab926586a805018f49720be31d681674f67c4a093409ddbc01a7533a"} Jul 02 19:48:59.701994 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.701968 2578 scope.go:117] "RemoveContainer" containerID="b29d1e8c2f50a740770730abc41c4f452a0b01cf6ad94666131ac1683ded838c" Jul 02 19:48:59.710835 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.710819 2578 scope.go:117] "RemoveContainer" containerID="25395cd58d328f2872e3c7603a684ecbae760b0247fe7996ffcf6147d5b2b11e" Jul 02 19:48:59.718767 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.718751 2578 scope.go:117] "RemoveContainer" containerID="b29d1e8c2f50a740770730abc41c4f452a0b01cf6ad94666131ac1683ded838c" Jul 02 19:48:59.719023 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:48:59.719006 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b29d1e8c2f50a740770730abc41c4f452a0b01cf6ad94666131ac1683ded838c\": container with ID starting with b29d1e8c2f50a740770730abc41c4f452a0b01cf6ad94666131ac1683ded838c not found: ID does not exist" containerID="b29d1e8c2f50a740770730abc41c4f452a0b01cf6ad94666131ac1683ded838c" Jul 02 19:48:59.719092 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.719035 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b29d1e8c2f50a740770730abc41c4f452a0b01cf6ad94666131ac1683ded838c"} err="failed to get container status \"b29d1e8c2f50a740770730abc41c4f452a0b01cf6ad94666131ac1683ded838c\": rpc error: code = NotFound desc = could not find container \"b29d1e8c2f50a740770730abc41c4f452a0b01cf6ad94666131ac1683ded838c\": container with ID starting with b29d1e8c2f50a740770730abc41c4f452a0b01cf6ad94666131ac1683ded838c not found: ID does not exist" Jul 02 19:48:59.719092 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.719057 2578 scope.go:117] "RemoveContainer" containerID="25395cd58d328f2872e3c7603a684ecbae760b0247fe7996ffcf6147d5b2b11e" Jul 02 19:48:59.719271 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:48:59.719251 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25395cd58d328f2872e3c7603a684ecbae760b0247fe7996ffcf6147d5b2b11e\": container with ID starting with 25395cd58d328f2872e3c7603a684ecbae760b0247fe7996ffcf6147d5b2b11e not found: ID does not exist" containerID="25395cd58d328f2872e3c7603a684ecbae760b0247fe7996ffcf6147d5b2b11e" Jul 02 19:48:59.719368 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.719273 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25395cd58d328f2872e3c7603a684ecbae760b0247fe7996ffcf6147d5b2b11e"} err="failed to get container status \"25395cd58d328f2872e3c7603a684ecbae760b0247fe7996ffcf6147d5b2b11e\": rpc error: code = NotFound desc = could not find container \"25395cd58d328f2872e3c7603a684ecbae760b0247fe7996ffcf6147d5b2b11e\": container with ID starting with 25395cd58d328f2872e3c7603a684ecbae760b0247fe7996ffcf6147d5b2b11e not found: ID does not exist" Jul 02 19:48:59.727691 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.727673 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn"] Jul 02 19:48:59.727775 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.727712 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-67cf22-openshift-default-6f65f467f9-mw7wn"] Jul 02 19:48:59.740174 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.740155 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-token\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:48:59.740174 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.740174 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bac33b1-cbcb-4d80-a665-749c69797d04-istiod-ca-cert\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:48:59.740337 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.740182 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-workload-certs\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:48:59.740337 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.740190 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-podinfo\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:48:59.740337 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.740198 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bac33b1-cbcb-4d80-a665-749c69797d04-istio-envoy\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:48:59.740337 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:48:59.740206 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fg5ts\" (UniqueName: \"kubernetes.io/projected/6bac33b1-cbcb-4d80-a665-749c69797d04-kube-api-access-fg5ts\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:49:01.644221 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:01.644182 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4"] Jul 02 19:49:01.644662 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:01.644435 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" podUID="c95d159a-e242-4468-9a93-8068a46a5576" containerName="istio-proxy" containerID="cri-o://e0319fff412182d8ad80bde98116096b6f9e5b5de65db80fedf29a9ffeb248cb" gracePeriod=30 Jul 02 19:49:01.729906 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:01.729719 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bac33b1-cbcb-4d80-a665-749c69797d04" path="/var/lib/kubelet/pods/6bac33b1-cbcb-4d80-a665-749c69797d04/volumes" Jul 02 19:49:02.365684 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:02.365654 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c675c4ddf-trhtp_90ccc1f4-459f-49a8-9075-959f9fc619be/authorino/0.log" Jul 02 19:49:05.163678 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:05.163642 2578 patch_prober.go:28] interesting pod/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 02 19:49:05.164056 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:05.163698 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" podUID="c95d159a-e242-4468-9a93-8068a46a5576" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 19:49:06.905016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:06.904992 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:49:07.007841 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.007741 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-credential-socket\") pod \"c95d159a-e242-4468-9a93-8068a46a5576\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " Jul 02 19:49:07.007841 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.007791 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c95d159a-e242-4468-9a93-8068a46a5576-istio-token\") pod \"c95d159a-e242-4468-9a93-8068a46a5576\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " Jul 02 19:49:07.007841 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.007825 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cw9lb\" (UniqueName: \"kubernetes.io/projected/c95d159a-e242-4468-9a93-8068a46a5576-kube-api-access-cw9lb\") pod \"c95d159a-e242-4468-9a93-8068a46a5576\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " Jul 02 19:49:07.007841 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.007841 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-workload-certs\") pod \"c95d159a-e242-4468-9a93-8068a46a5576\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " Jul 02 19:49:07.008184 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.007875 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-istio-envoy\") pod \"c95d159a-e242-4468-9a93-8068a46a5576\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " Jul 02 19:49:07.008184 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.007890 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-istio-data\") pod \"c95d159a-e242-4468-9a93-8068a46a5576\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " Jul 02 19:49:07.008184 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.007922 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c95d159a-e242-4468-9a93-8068a46a5576-istio-podinfo\") pod \"c95d159a-e242-4468-9a93-8068a46a5576\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " Jul 02 19:49:07.008184 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.007955 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c95d159a-e242-4468-9a93-8068a46a5576-istiod-ca-cert\") pod \"c95d159a-e242-4468-9a93-8068a46a5576\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " Jul 02 19:49:07.008184 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.007983 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-workload-socket\") pod \"c95d159a-e242-4468-9a93-8068a46a5576\" (UID: \"c95d159a-e242-4468-9a93-8068a46a5576\") " Jul 02 19:49:07.008184 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.008065 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c95d159a-e242-4468-9a93-8068a46a5576" (UID: "c95d159a-e242-4468-9a93-8068a46a5576"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:49:07.008539 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.008286 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-credential-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:49:07.008539 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.008526 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c95d159a-e242-4468-9a93-8068a46a5576" (UID: "c95d159a-e242-4468-9a93-8068a46a5576"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:49:07.008669 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.008465 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c95d159a-e242-4468-9a93-8068a46a5576-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c95d159a-e242-4468-9a93-8068a46a5576" (UID: "c95d159a-e242-4468-9a93-8068a46a5576"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:49:07.008669 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.008544 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c95d159a-e242-4468-9a93-8068a46a5576" (UID: "c95d159a-e242-4468-9a93-8068a46a5576"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:49:07.008931 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.008903 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-istio-data" (OuterVolumeSpecName: "istio-data") pod "c95d159a-e242-4468-9a93-8068a46a5576" (UID: "c95d159a-e242-4468-9a93-8068a46a5576"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:49:07.010237 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.010199 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c95d159a-e242-4468-9a93-8068a46a5576" (UID: "c95d159a-e242-4468-9a93-8068a46a5576"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:49:07.010346 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.010277 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c95d159a-e242-4468-9a93-8068a46a5576-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c95d159a-e242-4468-9a93-8068a46a5576" (UID: "c95d159a-e242-4468-9a93-8068a46a5576"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 19:49:07.010346 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.010296 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c95d159a-e242-4468-9a93-8068a46a5576-kube-api-access-cw9lb" (OuterVolumeSpecName: "kube-api-access-cw9lb") pod "c95d159a-e242-4468-9a93-8068a46a5576" (UID: "c95d159a-e242-4468-9a93-8068a46a5576"). InnerVolumeSpecName "kube-api-access-cw9lb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:49:07.010346 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.010325 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c95d159a-e242-4468-9a93-8068a46a5576-istio-token" (OuterVolumeSpecName: "istio-token") pod "c95d159a-e242-4468-9a93-8068a46a5576" (UID: "c95d159a-e242-4468-9a93-8068a46a5576"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:49:07.109131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.109096 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cw9lb\" (UniqueName: \"kubernetes.io/projected/c95d159a-e242-4468-9a93-8068a46a5576-kube-api-access-cw9lb\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:49:07.109131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.109126 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-workload-certs\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:49:07.109131 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.109136 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-istio-envoy\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:49:07.109383 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.109143 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-istio-data\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:49:07.109383 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.109151 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c95d159a-e242-4468-9a93-8068a46a5576-istio-podinfo\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:49:07.109383 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.109158 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c95d159a-e242-4468-9a93-8068a46a5576-istiod-ca-cert\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:49:07.109383 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.109166 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95d159a-e242-4468-9a93-8068a46a5576-workload-socket\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:49:07.109383 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.109175 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c95d159a-e242-4468-9a93-8068a46a5576-istio-token\") on node \"ip-10-0-133-161.ec2.internal\" DevicePath \"\"" Jul 02 19:49:07.165737 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.165706 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kzj2c_66139e53-0f64-4c2a-9fb1-bbf4b6267f35/manager/0.log" Jul 02 19:49:07.286839 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.286750 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-qhzkx_b175954f-c460-4009-8bb7-40d192370efb/manager/0.log" Jul 02 19:49:07.745738 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.745704 2578 generic.go:356] "Generic (PLEG): container finished" podID="c95d159a-e242-4468-9a93-8068a46a5576" containerID="e0319fff412182d8ad80bde98116096b6f9e5b5de65db80fedf29a9ffeb248cb" exitCode=0 Jul 02 19:49:07.745866 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.745745 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" Jul 02 19:49:07.745866 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.745780 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" event={"ID":"c95d159a-e242-4468-9a93-8068a46a5576","Type":"ContainerDied","Data":"e0319fff412182d8ad80bde98116096b6f9e5b5de65db80fedf29a9ffeb248cb"} Jul 02 19:49:07.745866 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.745834 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4" event={"ID":"c95d159a-e242-4468-9a93-8068a46a5576","Type":"ContainerDied","Data":"3b55cee5e7cb7a0773c4c8dc7bb5737020cee16a295e8f6e1bfcef0d6b51db7b"} Jul 02 19:49:07.745976 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.745866 2578 scope.go:117] "RemoveContainer" containerID="e0319fff412182d8ad80bde98116096b6f9e5b5de65db80fedf29a9ffeb248cb" Jul 02 19:49:07.755122 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.755109 2578 scope.go:117] "RemoveContainer" containerID="394df70b1dd603d650efdd8fb3dc9a279fe4b4328ef855f386a2674845d8e558" Jul 02 19:49:07.763059 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.762913 2578 scope.go:117] "RemoveContainer" containerID="e0319fff412182d8ad80bde98116096b6f9e5b5de65db80fedf29a9ffeb248cb" Jul 02 19:49:07.763444 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:49:07.763422 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0319fff412182d8ad80bde98116096b6f9e5b5de65db80fedf29a9ffeb248cb\": container with ID starting with e0319fff412182d8ad80bde98116096b6f9e5b5de65db80fedf29a9ffeb248cb not found: ID does not exist" containerID="e0319fff412182d8ad80bde98116096b6f9e5b5de65db80fedf29a9ffeb248cb" Jul 02 19:49:07.763525 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.763451 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0319fff412182d8ad80bde98116096b6f9e5b5de65db80fedf29a9ffeb248cb"} err="failed to get container status \"e0319fff412182d8ad80bde98116096b6f9e5b5de65db80fedf29a9ffeb248cb\": rpc error: code = NotFound desc = could not find container \"e0319fff412182d8ad80bde98116096b6f9e5b5de65db80fedf29a9ffeb248cb\": container with ID starting with e0319fff412182d8ad80bde98116096b6f9e5b5de65db80fedf29a9ffeb248cb not found: ID does not exist" Jul 02 19:49:07.763525 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.763468 2578 scope.go:117] "RemoveContainer" containerID="394df70b1dd603d650efdd8fb3dc9a279fe4b4328ef855f386a2674845d8e558" Jul 02 19:49:07.763739 ip-10-0-133-161 kubenswrapper[2578]: E0702 19:49:07.763725 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"394df70b1dd603d650efdd8fb3dc9a279fe4b4328ef855f386a2674845d8e558\": container with ID starting with 394df70b1dd603d650efdd8fb3dc9a279fe4b4328ef855f386a2674845d8e558 not found: ID does not exist" containerID="394df70b1dd603d650efdd8fb3dc9a279fe4b4328ef855f386a2674845d8e558" Jul 02 19:49:07.763782 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.763742 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"394df70b1dd603d650efdd8fb3dc9a279fe4b4328ef855f386a2674845d8e558"} err="failed to get container status \"394df70b1dd603d650efdd8fb3dc9a279fe4b4328ef855f386a2674845d8e558\": rpc error: code = NotFound desc = could not find container \"394df70b1dd603d650efdd8fb3dc9a279fe4b4328ef855f386a2674845d8e558\": container with ID starting with 394df70b1dd603d650efdd8fb3dc9a279fe4b4328ef855f386a2674845d8e558 not found: ID does not exist" Jul 02 19:49:07.765838 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.765815 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4"] Jul 02 19:49:07.768516 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.768498 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-0c31dc-openshift-default-7578464f9f-kx2t4"] Jul 02 19:49:07.886004 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:07.885975 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rf6t5_1931d911-68dd-48ac-836d-b57761b26166/server/0.log" Jul 02 19:49:08.013477 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:08.012871 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fw6d8_6316e3d6-05d3-46e3-8fe2-986e631c898b/manager/0.log" Jul 02 19:49:08.409561 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:08.409537 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8ddf44b5-szh7t_6439ecfe-740b-423e-b6d7-fd44ad306c87/copy-manifests/0.log" Jul 02 19:49:08.416793 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:08.416769 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8ddf44b5-szh7t_6439ecfe-740b-423e-b6d7-fd44ad306c87/manager/0.log" Jul 02 19:49:09.343417 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:09.343370 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j_7da25c73-a292-4803-8fa5-58d9773329dd/util/0.log" Jul 02 19:49:09.357601 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:09.357580 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j_7da25c73-a292-4803-8fa5-58d9773329dd/pull/0.log" Jul 02 19:49:09.368725 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:09.368705 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j_7da25c73-a292-4803-8fa5-58d9773329dd/extract/0.log" Jul 02 19:49:09.483052 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:09.483023 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4_b8912423-b85b-47fc-8969-9ca35fa01f7b/util/0.log" Jul 02 19:49:09.494256 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:09.494233 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4_b8912423-b85b-47fc-8969-9ca35fa01f7b/pull/0.log" Jul 02 19:49:09.502905 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:09.502866 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4_b8912423-b85b-47fc-8969-9ca35fa01f7b/extract/0.log" Jul 02 19:49:09.617509 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:09.617422 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k_a6a1bfe1-7c8e-400d-ad31-59fab76a5611/util/0.log" Jul 02 19:49:09.625496 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:09.625468 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k_a6a1bfe1-7c8e-400d-ad31-59fab76a5611/pull/0.log" Jul 02 19:49:09.633114 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:09.633086 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k_a6a1bfe1-7c8e-400d-ad31-59fab76a5611/extract/0.log" Jul 02 19:49:09.727197 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:09.727163 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c95d159a-e242-4468-9a93-8068a46a5576" path="/var/lib/kubelet/pods/c95d159a-e242-4468-9a93-8068a46a5576/volumes" Jul 02 19:49:09.753210 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:09.753181 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5_e0131c51-c13a-4538-aa05-16171908f1bc/pull/0.log" Jul 02 19:49:09.760263 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:09.760236 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5_e0131c51-c13a-4538-aa05-16171908f1bc/extract/0.log" Jul 02 19:49:09.767634 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:09.767617 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5_e0131c51-c13a-4538-aa05-16171908f1bc/util/0.log" Jul 02 19:49:09.881591 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:09.881514 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c675c4ddf-trhtp_90ccc1f4-459f-49a8-9075-959f9fc619be/authorino/0.log" Jul 02 19:49:10.004365 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:10.004333 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-h7dk4_c955b662-ccce-46c9-9773-13c92868a7c8/manager/0.log" Jul 02 19:49:10.119075 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:10.119045 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-n9txz_2c42f01d-d3bb-47b2-8944-ffd9d0838c89/manager/0.log" Jul 02 19:49:10.233821 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:10.233748 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-mhxr8_4e241b2c-f017-4bb6-8fe3-400a3f3ec955/kuadrant-console-plugin/0.log" Jul 02 19:49:10.352889 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:10.352862 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-g45f6_7746eda0-2e7a-4e63-b8c8-80876606bbc4/registry-server/0.log" Jul 02 19:49:10.472329 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:10.472297 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-gcbgd_b22ad585-c1dd-4194-a7a0-f677e1e903cf/manager/0.log" Jul 02 19:49:10.707846 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:10.707804 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-xvk7x_ef79892f-04af-465b-9a83-aa15d89925dc/manager/0.log" Jul 02 19:49:11.079756 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:11.079731 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678wgddt_2ee861e5-4888-4347-9b0f-65974149e684/istio-proxy/0.log" Jul 02 19:49:11.319302 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:11.319272 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bdf57ddb6-4v27p_1c0bbba4-27e1-4715-80d5-e5e157e52e7c/kube-auth-proxy/0.log" Jul 02 19:49:11.559286 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:11.559263 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-n2ds6_5d305985-0721-40bc-85bc-bcd412ab870c/istio-proxy/1.log" Jul 02 19:49:11.931016 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:11.930941 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74f4b96cdc-gnnfv_9998d8a2-ea82-4b1f-aae9-554e79db6264/router/0.log" Jul 02 19:49:12.387022 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:12.386994 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-lg2t7_c5e2a2d0-a12e-4810-bae1-96e0b166df00/main/0.log" Jul 02 19:49:12.638281 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:12.638210 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnqcsb_57b4d1f9-ebc3-4988-beb9-ed30f60daae2/main/0.log" Jul 02 19:49:19.582602 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:19.582572 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p2dbb_7bd39ca6-f010-4e22-b7b9-04bf2acece57/global-pull-secret-syncer/0.log" Jul 02 19:49:19.676759 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:19.676733 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vqm46_798f8da0-e51d-47d2-b055-6456e8e9c1e1/konnectivity-agent/0.log" Jul 02 19:49:19.764781 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:19.764755 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-161.ec2.internal_b3d1df1679904c800dfb191bda1f933a/haproxy/0.log" Jul 02 19:49:24.016010 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.015982 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j_7da25c73-a292-4803-8fa5-58d9773329dd/extract/0.log" Jul 02 19:49:24.040262 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.040238 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j_7da25c73-a292-4803-8fa5-58d9773329dd/util/0.log" Jul 02 19:49:24.065004 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.064983 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vzf8j_7da25c73-a292-4803-8fa5-58d9773329dd/pull/0.log" Jul 02 19:49:24.099036 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.099014 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4_b8912423-b85b-47fc-8969-9ca35fa01f7b/extract/0.log" Jul 02 19:49:24.123902 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.123881 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4_b8912423-b85b-47fc-8969-9ca35fa01f7b/util/0.log" Jul 02 19:49:24.157752 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.157735 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02cgc4_b8912423-b85b-47fc-8969-9ca35fa01f7b/pull/0.log" Jul 02 19:49:24.192820 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.192802 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k_a6a1bfe1-7c8e-400d-ad31-59fab76a5611/extract/0.log" Jul 02 19:49:24.225936 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.225918 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k_a6a1bfe1-7c8e-400d-ad31-59fab76a5611/util/0.log" Jul 02 19:49:24.254051 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.254035 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m2d2k_a6a1bfe1-7c8e-400d-ad31-59fab76a5611/pull/0.log" Jul 02 19:49:24.285497 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.285443 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5_e0131c51-c13a-4538-aa05-16171908f1bc/extract/0.log" Jul 02 19:49:24.312809 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.312783 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5_e0131c51-c13a-4538-aa05-16171908f1bc/util/0.log" Jul 02 19:49:24.334610 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.334593 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z9tk5_e0131c51-c13a-4538-aa05-16171908f1bc/pull/0.log" Jul 02 19:49:24.592288 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.592258 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c675c4ddf-trhtp_90ccc1f4-459f-49a8-9075-959f9fc619be/authorino/0.log" Jul 02 19:49:24.630055 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.630030 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-h7dk4_c955b662-ccce-46c9-9773-13c92868a7c8/manager/0.log" Jul 02 19:49:24.654513 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.654493 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-n9txz_2c42f01d-d3bb-47b2-8944-ffd9d0838c89/manager/0.log" Jul 02 19:49:24.679376 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.679358 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-mhxr8_4e241b2c-f017-4bb6-8fe3-400a3f3ec955/kuadrant-console-plugin/0.log" Jul 02 19:49:24.718025 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.717999 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-g45f6_7746eda0-2e7a-4e63-b8c8-80876606bbc4/registry-server/0.log" Jul 02 19:49:24.825966 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.825937 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-gcbgd_b22ad585-c1dd-4194-a7a0-f677e1e903cf/manager/0.log" Jul 02 19:49:24.928834 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:24.928756 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-xvk7x_ef79892f-04af-465b-9a83-aa15d89925dc/manager/0.log" Jul 02 19:49:26.771183 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:26.771154 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-mlg8q_c8db16c6-0045-462c-a896-b66e91f4beb1/cluster-monitoring-operator/0.log" Jul 02 19:49:26.940102 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:26.940032 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fvcpp_93442352-1882-45bc-b2f9-e0ee72908d12/node-exporter/0.log" Jul 02 19:49:26.965663 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:26.965641 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fvcpp_93442352-1882-45bc-b2f9-e0ee72908d12/kube-rbac-proxy/0.log" Jul 02 19:49:26.987518 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:26.987498 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fvcpp_93442352-1882-45bc-b2f9-e0ee72908d12/init-textfile/0.log" Jul 02 19:49:29.110514 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.110470 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-fng8v_9d749313-98ec-4c39-9421-2f888d58af9c/networking-console-plugin/0.log" Jul 02 19:49:29.700826 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.700795 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj"] Jul 02 19:49:29.701169 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.701157 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bac33b1-cbcb-4d80-a665-749c69797d04" containerName="istio-proxy" Jul 02 19:49:29.701215 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.701170 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bac33b1-cbcb-4d80-a665-749c69797d04" containerName="istio-proxy" Jul 02 19:49:29.701215 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.701194 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c95d159a-e242-4468-9a93-8068a46a5576" containerName="istio-proxy" Jul 02 19:49:29.701215 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.701199 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c95d159a-e242-4468-9a93-8068a46a5576" containerName="istio-proxy" Jul 02 19:49:29.701215 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.701215 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c95d159a-e242-4468-9a93-8068a46a5576" containerName="istio-proxy" Jul 02 19:49:29.701348 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.701221 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c95d159a-e242-4468-9a93-8068a46a5576" containerName="istio-proxy" Jul 02 19:49:29.701348 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.701292 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c95d159a-e242-4468-9a93-8068a46a5576" containerName="istio-proxy" Jul 02 19:49:29.701348 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.701300 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bac33b1-cbcb-4d80-a665-749c69797d04" containerName="istio-proxy" Jul 02 19:49:29.704580 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.704562 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:29.707115 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.707094 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ll6bw\"/\"kube-root-ca.crt\"" Jul 02 19:49:29.707227 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.707133 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ll6bw\"/\"openshift-service-ca.crt\"" Jul 02 19:49:29.707681 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.707668 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ll6bw\"/\"default-dockercfg-9shf6\"" Jul 02 19:49:29.714972 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.714948 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj"] Jul 02 19:49:29.791480 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.791451 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e6b41280-3791-4849-800f-9545066fb007-podres\") pod \"perf-node-gather-daemonset-gcwcj\" (UID: \"e6b41280-3791-4849-800f-9545066fb007\") " pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:29.791618 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.791495 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e6b41280-3791-4849-800f-9545066fb007-proc\") pod \"perf-node-gather-daemonset-gcwcj\" (UID: \"e6b41280-3791-4849-800f-9545066fb007\") " pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:29.791618 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.791516 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq5kl\" (UniqueName: \"kubernetes.io/projected/e6b41280-3791-4849-800f-9545066fb007-kube-api-access-hq5kl\") pod \"perf-node-gather-daemonset-gcwcj\" (UID: \"e6b41280-3791-4849-800f-9545066fb007\") " pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:29.791745 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.791614 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e6b41280-3791-4849-800f-9545066fb007-lib-modules\") pod \"perf-node-gather-daemonset-gcwcj\" (UID: \"e6b41280-3791-4849-800f-9545066fb007\") " pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:29.791745 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.791664 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6b41280-3791-4849-800f-9545066fb007-sys\") pod \"perf-node-gather-daemonset-gcwcj\" (UID: \"e6b41280-3791-4849-800f-9545066fb007\") " pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:29.893025 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.892995 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e6b41280-3791-4849-800f-9545066fb007-podres\") pod \"perf-node-gather-daemonset-gcwcj\" (UID: \"e6b41280-3791-4849-800f-9545066fb007\") " pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:29.893174 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.893039 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e6b41280-3791-4849-800f-9545066fb007-proc\") pod \"perf-node-gather-daemonset-gcwcj\" (UID: \"e6b41280-3791-4849-800f-9545066fb007\") " pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:29.893174 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.893069 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq5kl\" (UniqueName: \"kubernetes.io/projected/e6b41280-3791-4849-800f-9545066fb007-kube-api-access-hq5kl\") pod \"perf-node-gather-daemonset-gcwcj\" (UID: \"e6b41280-3791-4849-800f-9545066fb007\") " pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:29.893174 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.893113 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e6b41280-3791-4849-800f-9545066fb007-lib-modules\") pod \"perf-node-gather-daemonset-gcwcj\" (UID: \"e6b41280-3791-4849-800f-9545066fb007\") " pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:29.893174 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.893141 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e6b41280-3791-4849-800f-9545066fb007-proc\") pod \"perf-node-gather-daemonset-gcwcj\" (UID: \"e6b41280-3791-4849-800f-9545066fb007\") " pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:29.893174 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.893145 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6b41280-3791-4849-800f-9545066fb007-sys\") pod \"perf-node-gather-daemonset-gcwcj\" (UID: \"e6b41280-3791-4849-800f-9545066fb007\") " pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:29.893450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.893185 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6b41280-3791-4849-800f-9545066fb007-sys\") pod \"perf-node-gather-daemonset-gcwcj\" (UID: \"e6b41280-3791-4849-800f-9545066fb007\") " pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:29.893450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.893164 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e6b41280-3791-4849-800f-9545066fb007-podres\") pod \"perf-node-gather-daemonset-gcwcj\" (UID: \"e6b41280-3791-4849-800f-9545066fb007\") " pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:29.893450 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.893269 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e6b41280-3791-4849-800f-9545066fb007-lib-modules\") pod \"perf-node-gather-daemonset-gcwcj\" (UID: \"e6b41280-3791-4849-800f-9545066fb007\") " pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:29.900951 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:29.900932 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq5kl\" (UniqueName: \"kubernetes.io/projected/e6b41280-3791-4849-800f-9545066fb007-kube-api-access-hq5kl\") pod \"perf-node-gather-daemonset-gcwcj\" (UID: \"e6b41280-3791-4849-800f-9545066fb007\") " pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:30.015565 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:30.015507 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:30.144974 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:30.144949 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj"] Jul 02 19:49:30.146947 ip-10-0-133-161 kubenswrapper[2578]: W0702 19:49:30.146919 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode6b41280_3791_4849_800f_9545066fb007.slice/crio-022a28712497424772bf7bc2c246ac0b2a094898a1a5d9e3c2199c3c96e0f7e8 WatchSource:0}: Error finding container 022a28712497424772bf7bc2c246ac0b2a094898a1a5d9e3c2199c3c96e0f7e8: Status 404 returned error can't find the container with id 022a28712497424772bf7bc2c246ac0b2a094898a1a5d9e3c2199c3c96e0f7e8 Jul 02 19:49:30.395849 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:30.395825 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-z9kqm_65e26e27-be9e-4f51-afb7-08a3aed67d9b/download-server/0.log" Jul 02 19:49:30.863666 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:30.863636 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" event={"ID":"e6b41280-3791-4849-800f-9545066fb007","Type":"ContainerStarted","Data":"391318e3a505a1650418d0958c86f7fc1be347e6a9f7dc46fb4062f9f0be2050"} Jul 02 19:49:30.863832 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:30.863684 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" event={"ID":"e6b41280-3791-4849-800f-9545066fb007","Type":"ContainerStarted","Data":"022a28712497424772bf7bc2c246ac0b2a094898a1a5d9e3c2199c3c96e0f7e8"} Jul 02 19:49:30.863832 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:30.863736 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:30.879451 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:30.879411 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" podStartSLOduration=1.879381143 podStartE2EDuration="1.879381143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:49:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:49:30.878430549 +0000 UTC m=+2513.681246009" watchObservedRunningTime="2026-07-02 19:49:30.879381143 +0000 UTC m=+2513.682196620" Jul 02 19:49:32.039458 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:32.039432 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-24ghv_b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71/dns/0.log" Jul 02 19:49:32.062378 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:32.062356 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-24ghv_b31b1e5b-c80b-47ca-b7f2-c9ad31a4ad71/kube-rbac-proxy/0.log" Jul 02 19:49:32.230531 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:32.230503 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jkjv7_8e01d2a6-a6aa-42e3-94b5-164e28d5a3cc/dns-node-resolver/0.log" Jul 02 19:49:32.800139 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:32.800107 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-57hcx_c0aec0a3-2014-4d88-a168-33b4b7048bb0/node-ca/0.log" Jul 02 19:49:33.834748 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:33.834716 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678wgddt_2ee861e5-4888-4347-9b0f-65974149e684/istio-proxy/0.log" Jul 02 19:49:34.055387 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:34.055359 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bdf57ddb6-4v27p_1c0bbba4-27e1-4715-80d5-e5e157e52e7c/kube-auth-proxy/0.log" Jul 02 19:49:34.112265 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:34.112177 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-n2ds6_5d305985-0721-40bc-85bc-bcd412ab870c/istio-proxy/0.log" Jul 02 19:49:34.138080 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:34.138052 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-n2ds6_5d305985-0721-40bc-85bc-bcd412ab870c/istio-proxy/1.log" Jul 02 19:49:34.282114 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:34.282072 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74f4b96cdc-gnnfv_9998d8a2-ea82-4b1f-aae9-554e79db6264/router/0.log" Jul 02 19:49:35.001499 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:35.001467 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-42jp6_9340dfab-6698-4817-82fa-e976787cfebb/serve-healthcheck-canary/0.log" Jul 02 19:49:35.555600 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:35.555570 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-xrtv2_2bb75416-de32-44da-b7ae-29678f1cc099/insights-operator/0.log" Jul 02 19:49:35.556804 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:35.556786 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-xrtv2_2bb75416-de32-44da-b7ae-29678f1cc099/insights-operator/1.log" Jul 02 19:49:35.689977 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:35.689954 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p47cx_d12a1023-4e31-4eb4-9aab-0b737eb11f51/kube-rbac-proxy/0.log" Jul 02 19:49:35.712722 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:35.712702 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p47cx_d12a1023-4e31-4eb4-9aab-0b737eb11f51/exporter/0.log" Jul 02 19:49:35.736472 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:35.736451 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p47cx_d12a1023-4e31-4eb4-9aab-0b737eb11f51/extractor/0.log" Jul 02 19:49:36.878827 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:36.878801 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ll6bw/perf-node-gather-daemonset-gcwcj" Jul 02 19:49:37.924780 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:37.924752 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kzj2c_66139e53-0f64-4c2a-9fb1-bbf4b6267f35/manager/0.log" Jul 02 19:49:38.048276 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:38.048252 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-qhzkx_b175954f-c460-4009-8bb7-40d192370efb/manager/0.log" Jul 02 19:49:38.333325 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:38.333297 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rf6t5_1931d911-68dd-48ac-836d-b57761b26166/server/0.log" Jul 02 19:49:38.404677 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:38.404650 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fw6d8_6316e3d6-05d3-46e3-8fe2-986e631c898b/manager/0.log" Jul 02 19:49:38.628154 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:38.628122 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8ddf44b5-szh7t_6439ecfe-740b-423e-b6d7-fd44ad306c87/manager/0.log" Jul 02 19:49:38.648638 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:38.648604 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8ddf44b5-szh7t_6439ecfe-740b-423e-b6d7-fd44ad306c87/copy-manifests/0.log" Jul 02 19:49:40.418881 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:40.418853 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678wgddt_2ee861e5-4888-4347-9b0f-65974149e684/istio-proxy/0.log" Jul 02 19:49:40.630869 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:40.630836 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bdf57ddb6-4v27p_1c0bbba4-27e1-4715-80d5-e5e157e52e7c/kube-auth-proxy/0.log" Jul 02 19:49:40.677021 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:40.676948 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-n2ds6_5d305985-0721-40bc-85bc-bcd412ab870c/istio-proxy/0.log" Jul 02 19:49:40.702235 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:40.702210 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-n2ds6_5d305985-0721-40bc-85bc-bcd412ab870c/istio-proxy/1.log" Jul 02 19:49:40.830144 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:40.830120 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74f4b96cdc-gnnfv_9998d8a2-ea82-4b1f-aae9-554e79db6264/router/0.log" Jul 02 19:49:42.083912 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:42.083887 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678wgddt_2ee861e5-4888-4347-9b0f-65974149e684/istio-proxy/0.log" Jul 02 19:49:42.295080 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:42.295056 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bdf57ddb6-4v27p_1c0bbba4-27e1-4715-80d5-e5e157e52e7c/kube-auth-proxy/0.log" Jul 02 19:49:42.336557 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:42.336465 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-n2ds6_5d305985-0721-40bc-85bc-bcd412ab870c/istio-proxy/0.log" Jul 02 19:49:42.365175 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:42.365152 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-n2ds6_5d305985-0721-40bc-85bc-bcd412ab870c/istio-proxy/1.log" Jul 02 19:49:42.489893 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:42.489865 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74f4b96cdc-gnnfv_9998d8a2-ea82-4b1f-aae9-554e79db6264/router/0.log" Jul 02 19:49:45.332832 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:45.332804 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-5kzdc_317197ef-bfcd-4aca-9484-95d9b7d15183/migrator/0.log" Jul 02 19:49:45.358827 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:45.358798 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-5kzdc_317197ef-bfcd-4aca-9484-95d9b7d15183/graceful-termination/0.log" Jul 02 19:49:47.029800 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:47.029774 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prfg6_365022fe-6d5e-4e74-be92-b791e81692b2/kube-multus-additional-cni-plugins/0.log" Jul 02 19:49:47.065971 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:47.065947 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prfg6_365022fe-6d5e-4e74-be92-b791e81692b2/egress-router-binary-copy/0.log" Jul 02 19:49:47.098282 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:47.098262 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prfg6_365022fe-6d5e-4e74-be92-b791e81692b2/cni-plugins/0.log" Jul 02 19:49:47.129804 ip-10-0-133-161 kubenswrapper[2578]: I0702 19:49:47.129785 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prfg6_365022fe-6d5e-4e74-be92-b791e81692b2/bond-cni-plugin/0.log"