Jul 22 02:19:28.971133 ip-10-0-132-119 systemd[1]: Starting Kubernetes Kubelet... Jul 22 02:19:29.406168 ip-10-0-132-119 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 02:19:29.406168 ip-10-0-132-119 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 02:19:29.406168 ip-10-0-132-119 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 02:19:29.406168 ip-10-0-132-119 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 02:19:29.406168 ip-10-0-132-119 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 02:19:29.407174 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.407063 2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 02:19:29.412061 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412043 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:19:29.412061 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412061 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412067 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412070 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412073 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412077 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412080 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412083 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412086 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412089 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412091 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412094 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412096 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412099 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412101 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412104 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412107 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412109 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412112 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412115 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412117 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:19:29.412131 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412126 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412129 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412131 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412133 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412136 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412139 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412141 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412144 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412147 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412149 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412152 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412154 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412157 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412159 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412163 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412167 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412170 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412172 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412175 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412178 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:19:29.412629 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412182 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412185 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412188 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412191 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412193 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412196 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412199 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412203 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412207 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412210 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412213 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412215 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412217 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412220 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412223 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412226 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412229 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412232 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412235 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:19:29.413112 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412237 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412239 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412242 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412244 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412247 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412249 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412253 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412255 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412258 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412260 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412263 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412265 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412268 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412270 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412273 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412275 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412278 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412280 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412282 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412284 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:19:29.413590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412287 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412289 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412292 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412294 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412296 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412299 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412302 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412305 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412308 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412310 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412314 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412316 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412318 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412321 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412323 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412325 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412328 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412330 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.412332 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413090 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:19:29.414078 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413097 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413101 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413104 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413106 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413109 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413112 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413115 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413118 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413121 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413123 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413126 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413129 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413131 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413133 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413136 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413138 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413142 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413145 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413148 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:19:29.414590 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413151 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413155 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413157 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413160 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413163 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413165 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413167 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413170 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413173 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413175 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413178 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413180 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413182 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413185 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413188 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413191 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413194 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413196 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413198 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413201 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:19:29.415068 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413203 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413206 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413208 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413210 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413213 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413215 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413217 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413220 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413223 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413227 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413230 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413232 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413236 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413239 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413242 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413245 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413247 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413250 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413252 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:19:29.415582 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413255 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413257 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413259 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413261 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413265 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413268 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413272 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413274 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413277 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413280 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413282 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413284 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413287 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413289 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413292 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413294 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413296 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413299 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413301 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:19:29.416054 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413303 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413306 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413308 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413310 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413313 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413316 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413319 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413321 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413325 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413327 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413330 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413332 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413334 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413337 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413339 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413342 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413344 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413347 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413349 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413351 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:19:29.416506 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413354 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413745 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413750 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413753 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413757 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413762 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413766 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413770 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413775 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413779 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413834 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413841 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413848 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413852 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413856 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413859 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413862 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413866 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413869 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:19:29.417007 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413872 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413875 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413878 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413881 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413883 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413886 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413888 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413890 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413893 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413896 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413898 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413901 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413904 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413907 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413913 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413915 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413919 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413921 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413924 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413926 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:19:29.417468 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413929 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413931 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413934 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413937 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413939 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413942 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413947 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413949 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413952 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413955 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413957 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413960 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413963 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413965 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413968 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413970 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413973 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413976 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413978 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413981 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:19:29.417956 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413983 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413986 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413989 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413992 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413995 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.413998 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414003 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414007 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414011 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414014 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414017 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414020 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414023 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414025 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414028 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414034 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414038 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414041 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414044 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414047 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:19:29.418433 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414050 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414052 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414055 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414058 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414061 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414063 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414066 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414068 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414070 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414073 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414076 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414078 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414081 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414083 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414086 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414089 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414091 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414094 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414105 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414108 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:19:29.418918 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414112 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414206 2557 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414214 2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414221 2557 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414226 2557 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414230 2557 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414234 2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414239 2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414243 2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414246 2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414249 2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414252 2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414255 2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414258 2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414261 2557 flags.go:64] FLAG: --cgroup-root="" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414264 2557 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414267 2557 flags.go:64] FLAG: --client-ca-file="" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414270 2557 flags.go:64] FLAG: --cloud-provider="external" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414273 2557 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414277 2557 flags.go:64] FLAG: --cluster-domain="" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414280 2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414282 2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414285 2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414289 2557 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 02:19:29.419382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414292 2557 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414295 2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414298 2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414301 2557 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414304 2557 flags.go:64] FLAG: --contention-profiling="false" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414306 2557 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414311 2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414314 2557 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414317 2557 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414322 2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414325 2557 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414327 2557 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414331 2557 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414334 2557 flags.go:64] FLAG: --enable-server="true" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414337 2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414344 2557 flags.go:64] FLAG: --event-burst="100" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414347 2557 flags.go:64] FLAG: --event-qps="50" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414350 2557 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414353 2557 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414356 2557 flags.go:64] FLAG: --eviction-hard="" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414360 2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414363 2557 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414366 2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414369 2557 flags.go:64] FLAG: --eviction-soft="" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414372 2557 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 02:19:29.419995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414374 2557 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414377 2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414380 2557 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414383 2557 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414386 2557 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414388 2557 flags.go:64] FLAG: --feature-gates="" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414392 2557 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414395 2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414398 2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414401 2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414403 2557 flags.go:64] FLAG: --healthz-port="10248" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414406 2557 flags.go:64] FLAG: --help="false" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414409 2557 flags.go:64] FLAG: --hostname-override="ip-10-0-132-119.ec2.internal" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414413 2557 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414416 2557 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414419 2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414422 2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414426 2557 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414429 2557 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414432 2557 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414434 2557 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414437 2557 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414441 2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414444 2557 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 02:19:29.420600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414447 2557 flags.go:64] FLAG: --kube-reserved="" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414450 2557 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414452 2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414455 2557 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414458 2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414461 2557 flags.go:64] FLAG: --lock-file="" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414463 2557 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414466 2557 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414469 2557 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414475 2557 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414477 2557 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414480 2557 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414483 2557 flags.go:64] FLAG: --logging-format="text" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414486 2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414489 2557 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414491 2557 flags.go:64] FLAG: --manifest-url="" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414494 2557 flags.go:64] FLAG: --manifest-url-header="" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414499 2557 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414501 2557 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414505 2557 flags.go:64] FLAG: --max-pods="110" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414508 2557 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414512 2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414515 2557 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414518 2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414521 2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 02:19:29.421200 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414524 2557 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414528 2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414536 2557 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414539 2557 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414542 2557 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414546 2557 flags.go:64] FLAG: --pod-cidr="" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414549 2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414555 2557 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414558 2557 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414561 2557 flags.go:64] FLAG: --pods-per-core="0" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414563 2557 flags.go:64] FLAG: --port="10250" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414566 2557 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414569 2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f35bfcf5a4433c40" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414572 2557 flags.go:64] FLAG: --qos-reserved="" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414575 2557 flags.go:64] FLAG: --read-only-port="10255" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414578 2557 flags.go:64] FLAG: --register-node="true" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414581 2557 flags.go:64] FLAG: --register-with-taints="" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414585 2557 flags.go:64] FLAG: --registry-burst="10" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414587 2557 flags.go:64] FLAG: --registry-qps="5" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414590 2557 flags.go:64] FLAG: --reserved-cpus="" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414593 2557 flags.go:64] FLAG: --reserved-memory="" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414598 2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414601 2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414604 2557 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414607 2557 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 02:19:29.422090 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414609 2557 flags.go:64] FLAG: --runonce="false" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414612 2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414615 2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414619 2557 flags.go:64] FLAG: --seccomp-default="false" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414622 2557 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414625 2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414628 2557 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414631 2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414634 2557 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414638 2557 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414641 2557 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414643 2557 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414646 2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414650 2557 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414653 2557 flags.go:64] FLAG: --system-cgroups="" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414655 2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414660 2557 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414663 2557 flags.go:64] FLAG: --tls-cert-file="" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414666 2557 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414670 2557 flags.go:64] FLAG: --tls-min-version="" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414672 2557 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414675 2557 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414678 2557 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414680 2557 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414683 2557 flags.go:64] FLAG: --v="2" Jul 22 02:19:29.423001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414687 2557 flags.go:64] FLAG: --version="false" Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414691 2557 flags.go:64] FLAG: --vmodule="" Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414694 2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.414698 2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414838 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414842 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414844 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414847 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414850 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414852 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414855 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414859 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414863 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414866 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414869 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414872 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414874 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414877 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414880 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:19:29.423716 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414883 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414887 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414889 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414892 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414894 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414896 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414899 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414901 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414904 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414906 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414909 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414911 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414914 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414916 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414920 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414923 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414926 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414929 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414932 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:19:29.424206 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414934 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414938 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414941 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414944 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414947 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414949 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414952 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414954 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414957 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414959 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414962 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414964 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414967 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414969 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414973 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414976 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414978 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414981 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414983 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:19:29.424669 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414985 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414988 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414991 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414993 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414996 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.414998 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415001 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415003 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415006 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415008 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415010 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415013 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415015 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415018 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415020 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415023 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415025 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415028 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415031 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415033 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415035 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:19:29.425255 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415038 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415041 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415043 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415045 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415048 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415051 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415054 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415057 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415059 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415061 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415064 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415066 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415069 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415071 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415074 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415076 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415079 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415081 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415084 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415086 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:19:29.425745 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415089 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:19:29.426251 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415091 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:19:29.426251 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415094 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:19:29.426251 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415096 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:19:29.426251 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.415098 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:19:29.426251 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.415844 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 02:19:29.426251 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.424463 2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 02:19:29.426251 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.424480 2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 02:19:29.426251 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.424500 2557 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 02:19:29.426251 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.424506 2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 02:19:29.426251 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424560 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:19:29.426251 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424566 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:19:29.426251 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424571 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:19:29.426251 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424576 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:19:29.426251 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424579 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424581 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424584 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424587 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424590 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424592 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424596 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424600 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424603 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424606 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424609 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424611 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424614 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424616 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424619 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424622 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424624 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424627 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424629 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:19:29.426603 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424632 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424635 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424638 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424640 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424643 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424645 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424649 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424651 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424654 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424657 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424659 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424662 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424664 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424667 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424669 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424672 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424674 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424677 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424680 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424683 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:19:29.427082 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424685 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424688 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424690 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424693 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424696 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424698 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424700 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424703 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424705 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424708 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424710 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424713 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424715 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424718 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424720 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424722 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424725 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424728 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424730 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424733 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:19:29.427560 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424735 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424737 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424740 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424743 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424745 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424748 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424750 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424753 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424755 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424759 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424762 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424764 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424766 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424769 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424771 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424774 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424777 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424779 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424782 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424784 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:19:29.428058 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424786 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:19:29.428577 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424789 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:19:29.428577 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424791 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:19:29.428577 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424793 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:19:29.428577 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424796 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:19:29.428577 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424798 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:19:29.428577 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424800 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:19:29.428577 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424802 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:19:29.428577 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424805 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:19:29.428577 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424808 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:19:29.428577 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424810 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:19:29.428577 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424813 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:19:29.428577 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424815 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:19:29.428577 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424817 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:19:29.428577 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424839 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:19:29.428577 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424846 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:19:29.428947 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.424851 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 02:19:29.428947 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424964 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:19:29.428947 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424989 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:19:29.428947 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424992 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:19:29.428947 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424995 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:19:29.428947 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.424997 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:19:29.428947 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425000 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:19:29.428947 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425004 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:19:29.428947 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425008 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:19:29.428947 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425011 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:19:29.428947 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425014 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:19:29.428947 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425017 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:19:29.428947 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425021 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:19:29.428947 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425025 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425029 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425032 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425034 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425038 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425040 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425043 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425045 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425048 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425050 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425053 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425055 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425058 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425061 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425063 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425066 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425069 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425071 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425073 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425076 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:19:29.429291 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425078 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425081 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425083 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425085 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425088 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425090 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425092 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425095 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425097 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425099 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425102 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425104 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425107 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425110 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425113 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425115 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425118 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425120 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425123 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425125 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:19:29.429771 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425127 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425130 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425133 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425135 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425138 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425140 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425142 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425145 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425147 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425151 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425154 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425157 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425159 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425162 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425164 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425167 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425169 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425172 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425174 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:19:29.430342 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425176 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425179 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425181 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425184 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425186 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425188 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425191 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425194 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425196 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425199 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425201 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425203 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425206 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425208 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425210 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425213 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425215 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425217 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425220 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425225 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:19:29.430802 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425227 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:19:29.431285 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425229 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:19:29.431285 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425232 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:19:29.431285 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425234 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:19:29.431285 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425237 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:19:29.431285 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425240 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:19:29.431285 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425242 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:19:29.431285 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:29.425244 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:19:29.431285 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.425249 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 02:19:29.431285 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.425369 2557 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 02:19:29.431285 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.428499 2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 02:19:29.431285 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.429911 2557 server.go:1019] "Starting client certificate rotation" Jul 22 02:19:29.431285 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.430017 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 02:19:29.431285 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.430054 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 02:19:29.432807 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.432792 2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 02:19:29.434791 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.434769 2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 02:19:29.444837 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.444805 2557 log.go:25] "Validated CRI v1 runtime API" Jul 22 02:19:29.448258 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.448244 2557 log.go:25] "Validated CRI v1 image API" Jul 22 02:19:29.450527 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.450513 2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 02:19:29.453481 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.453425 2557 fs.go:135] Filesystem UUIDs: map[16c5d7ae-0621-4b0b-a4a3-bb931fcc5443:/dev/nvme0n1p3 4036d42d-5b8b-4d8a-b14c-64b4ea68f652:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 22 02:19:29.453568 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.453480 2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 02:19:29.459479 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.459365 2557 manager.go:216] Machine: {Timestamp:2026-07-22 02:19:29.457893768 +0000 UTC m=+0.366758754 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3092189 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b3a5f9ab383470c53a6d01160f264 SystemUUID:ec2b3a5f-9ab3-8347-0c53-a6d01160f264 BootID:8443b047-071d-441c-a1c1-c8c3c831b0a3 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0c:97:a3:b1:dd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0c:97:a3:b1:dd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:f9:f4:45:f2:84 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 02:19:29.459479 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.459468 2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 02:19:29.459607 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.459555 2557 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 02:19:29.461799 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.461781 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 02:19:29.463171 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.463148 2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 02:19:29.463302 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.463174 2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-119.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 02:19:29.463980 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.463970 2557 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 02:19:29.464016 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.463981 2557 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 02:19:29.464016 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.463989 2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 02:19:29.464890 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.464880 2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 02:19:29.466616 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.466606 2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 02:19:29.470456 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.470445 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 22 02:19:29.470567 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.470558 2557 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 02:19:29.470602 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.470587 2557 kubelet.go:486] "Attempting to sync node with API server" Jul 22 02:19:29.470602 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.470597 2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 02:19:29.470656 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.470608 2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 02:19:29.470656 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.470616 2557 kubelet.go:398] "Adding apiserver pod source" Jul 22 02:19:29.470656 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.470626 2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 02:19:29.471048 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.471036 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 02:19:29.471100 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.471055 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 02:19:29.473547 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.473528 2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 02:19:29.475210 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.475197 2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 02:19:29.475261 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.475227 2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 02:19:29.475943 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.475914 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 02:19:29.475943 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.475932 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 02:19:29.475943 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.475938 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 02:19:29.475943 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.475943 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 02:19:29.475943 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.475951 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 02:19:29.476168 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.475956 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 02:19:29.476168 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.475994 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 02:19:29.476168 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.476001 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 02:19:29.476168 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.476008 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 02:19:29.476168 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.476014 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 02:19:29.476168 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.476029 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 02:19:29.476168 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.476037 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 02:19:29.476168 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.476059 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 02:19:29.476168 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.476065 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 02:19:29.479087 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.479073 2557 server.go:1268] "Started kubelet" Jul 22 02:19:29.479240 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.479180 2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 02:19:29.479477 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.479425 2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 02:19:29.479574 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.479492 2557 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 02:19:29.479953 ip-10-0-132-119 systemd[1]: Started Kubernetes Kubelet. Jul 22 02:19:29.480623 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.480011 2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-119.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 02:19:29.480901 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.480771 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-119.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 02:19:29.480901 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.480774 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 02:19:29.480901 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.480899 2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 02:19:29.481504 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.481488 2557 server.go:310] "Adding debug handlers to kubelet server" Jul 22 02:19:29.484071 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.484053 2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 02:19:29.484676 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.483554 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-119.ec2.internal.18c47b2942235aaa default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-119.ec2.internal,UID:ip-10-0-132-119.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-119.ec2.internal,},FirstTimestamp:2026-07-22 02:19:29.47904785 +0000 UTC m=+0.387912834,LastTimestamp:2026-07-22 02:19:29.47904785 +0000 UTC m=+0.387912834,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-119.ec2.internal,}" Jul 22 02:19:29.485483 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.485466 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zf9fn" Jul 22 02:19:29.493654 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.493638 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zf9fn" Jul 22 02:19:29.496529 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.496515 2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 02:19:29.496587 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.496535 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 02:19:29.496669 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.496651 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Jul 22 02:19:29.497229 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.497212 2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 02:19:29.497315 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.497232 2557 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 02:19:29.497315 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.497265 2557 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 02:19:29.497409 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.497380 2557 reconstruct.go:97] "Volume reconstruction finished" Jul 22 02:19:29.497409 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.497389 2557 reconciler.go:29] "Reconciler: start to sync state" Jul 22 02:19:29.497513 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.497469 2557 factory.go:55] Registering systemd factory Jul 22 02:19:29.497513 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.497485 2557 factory.go:223] Registration of the systemd container factory successfully Jul 22 02:19:29.497860 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.497835 2557 factory.go:153] Registering CRI-O factory Jul 22 02:19:29.498149 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.498126 2557 factory.go:223] Registration of the crio container factory successfully Jul 22 02:19:29.498225 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.498217 2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 02:19:29.498289 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.498246 2557 factory.go:103] Registering Raw factory Jul 22 02:19:29.498289 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.498262 2557 manager.go:1196] Started watching for new ooms in manager Jul 22 02:19:29.499116 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.499094 2557 manager.go:349] Starting recovery of all containers Jul 22 02:19:29.500749 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.500659 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 22 02:19:29.500749 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.500708 2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-119.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 22 02:19:29.511247 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.511232 2557 manager.go:354] Recovery completed Jul 22 02:19:29.515755 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.515738 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:19:29.519733 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.519715 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:19:29.519859 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.519750 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:19:29.519859 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.519761 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:19:29.520340 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.520324 2557 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 02:19:29.520340 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.520340 2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 02:19:29.520514 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.520351 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:19:29.520514 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.520360 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 22 02:19:29.521424 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.521407 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:19:29.521510 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.521431 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:19:29.521510 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.521441 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:19:29.521874 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.521862 2557 policy_none.go:49] "None policy: Start" Jul 22 02:19:29.521928 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.521881 2557 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 02:19:29.521928 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.521896 2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 02:19:29.523168 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.523155 2557 policy_none.go:47] "Start" Jul 22 02:19:29.569536 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.569388 2557 manager.go:337] "Starting Device Plugin manager" Jul 22 02:19:29.575119 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.569567 2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 02:19:29.575119 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.569583 2557 server.go:87] "Starting device plugin registration server" Jul 22 02:19:29.575119 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.569799 2557 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 02:19:29.575119 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.569808 2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 02:19:29.575119 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.569896 2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 02:19:29.575119 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.569964 2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 02:19:29.575119 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.569971 2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 02:19:29.575119 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.570481 2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 02:19:29.575119 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.570521 2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-119.ec2.internal\" not found" Jul 22 02:19:29.601246 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.601211 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 02:19:29.602489 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.602468 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 02:19:29.602607 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.602506 2557 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 02:19:29.602607 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.602532 2557 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 02:19:29.602607 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.602563 2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 02:19:29.604689 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.604668 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:19:29.670930 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.670844 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:19:29.672865 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.672848 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:19:29.672935 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.672879 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:19:29.672935 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.672889 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:19:29.672935 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.672915 2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-119.ec2.internal" Jul 22 02:19:29.681497 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.681478 2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-119.ec2.internal" Jul 22 02:19:29.681560 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.681502 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-119.ec2.internal\": node \"ip-10-0-132-119.ec2.internal\" not found" Jul 22 02:19:29.698783 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.698753 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Jul 22 02:19:29.702886 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.702871 2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal"] Jul 22 02:19:29.702939 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.702923 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:19:29.704406 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.704390 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:19:29.704482 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.704424 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:19:29.704482 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.704439 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:19:29.705704 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.705693 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:19:29.705851 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.705838 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Jul 22 02:19:29.705892 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.705866 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:19:29.706477 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.706461 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:19:29.706528 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.706492 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:19:29.706528 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.706503 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:19:29.706601 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.706461 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:19:29.706601 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.706572 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:19:29.706601 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.706584 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:19:29.708540 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.708522 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" Jul 22 02:19:29.708540 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.708545 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:19:29.710183 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.710167 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:19:29.710267 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.710194 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:19:29.710267 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.710206 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:19:29.732551 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.732525 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-119.ec2.internal\" not found" node="ip-10-0-132-119.ec2.internal" Jul 22 02:19:29.737071 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.737051 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-119.ec2.internal\" not found" node="ip-10-0-132-119.ec2.internal" Jul 22 02:19:29.798860 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.798813 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/47803356514fd5d371d5e4ac55377679-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal\" (UID: \"47803356514fd5d371d5e4ac55377679\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Jul 22 02:19:29.798860 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.798839 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Jul 22 02:19:29.799027 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.798869 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47803356514fd5d371d5e4ac55377679-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal\" (UID: \"47803356514fd5d371d5e4ac55377679\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Jul 22 02:19:29.799027 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.798894 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/11b830b0e1c88ebf098f99a124478494-config\") pod \"kube-apiserver-proxy-ip-10-0-132-119.ec2.internal\" (UID: \"11b830b0e1c88ebf098f99a124478494\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" Jul 22 02:19:29.899323 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.899290 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Jul 22 02:19:29.899323 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.899314 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/47803356514fd5d371d5e4ac55377679-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal\" (UID: \"47803356514fd5d371d5e4ac55377679\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Jul 22 02:19:29.899482 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.899346 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47803356514fd5d371d5e4ac55377679-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal\" (UID: \"47803356514fd5d371d5e4ac55377679\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Jul 22 02:19:29.899482 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.899352 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/47803356514fd5d371d5e4ac55377679-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal\" (UID: \"47803356514fd5d371d5e4ac55377679\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Jul 22 02:19:29.899482 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.899369 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/11b830b0e1c88ebf098f99a124478494-config\") pod \"kube-apiserver-proxy-ip-10-0-132-119.ec2.internal\" (UID: \"11b830b0e1c88ebf098f99a124478494\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" Jul 22 02:19:29.899482 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.899421 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/11b830b0e1c88ebf098f99a124478494-config\") pod \"kube-apiserver-proxy-ip-10-0-132-119.ec2.internal\" (UID: \"11b830b0e1c88ebf098f99a124478494\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" Jul 22 02:19:29.899482 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:29.899425 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47803356514fd5d371d5e4ac55377679-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal\" (UID: \"47803356514fd5d371d5e4ac55377679\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Jul 22 02:19:30.000005 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:29.999977 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Jul 22 02:19:30.035193 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:30.035159 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Jul 22 02:19:30.040208 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:30.040187 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" Jul 22 02:19:30.100911 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:30.100880 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Jul 22 02:19:30.201338 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:30.201307 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Jul 22 02:19:30.301776 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:30.301709 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Jul 22 02:19:30.388439 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:30.388409 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:19:30.402380 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:30.402351 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Jul 22 02:19:30.429554 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:30.429520 2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 02:19:30.430009 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:30.429680 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 02:19:30.430009 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:30.429710 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 02:19:30.502810 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:30.502779 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Jul 22 02:19:30.502993 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:30.502807 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 02:14:29 +0000 UTC" deadline="2028-01-29 02:14:07.365135207 +0000 UTC" Jul 22 02:19:30.502993 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:30.502855 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13343h54m36.862284252s" Jul 22 02:19:30.580215 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:30.580175 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47803356514fd5d371d5e4ac55377679.slice/crio-7af785d7d1c8a82ab028cce802d4654120288ea35e739bb919778d02bcf89b3a WatchSource:0}: Error finding container 7af785d7d1c8a82ab028cce802d4654120288ea35e739bb919778d02bcf89b3a: Status 404 returned error can't find the container with id 7af785d7d1c8a82ab028cce802d4654120288ea35e739bb919778d02bcf89b3a Jul 22 02:19:30.584350 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:30.584335 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:19:30.603416 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:30.603388 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Jul 22 02:19:30.606417 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:30.606375 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" event={"ID":"47803356514fd5d371d5e4ac55377679","Type":"ContainerStarted","Data":"7af785d7d1c8a82ab028cce802d4654120288ea35e739bb919778d02bcf89b3a"} Jul 22 02:19:30.653203 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:30.653174 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11b830b0e1c88ebf098f99a124478494.slice/crio-9e6f657e6d3aca790ea8383d5f10954fe7799e4041bf4f1d54e76c5a14665d76 WatchSource:0}: Error finding container 9e6f657e6d3aca790ea8383d5f10954fe7799e4041bf4f1d54e76c5a14665d76: Status 404 returned error can't find the container with id 9e6f657e6d3aca790ea8383d5f10954fe7799e4041bf4f1d54e76c5a14665d76 Jul 22 02:19:30.690366 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:30.690344 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:19:30.697580 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:30.697561 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" Jul 22 02:19:30.707701 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:30.707679 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 02:19:30.708738 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:30.708725 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Jul 22 02:19:30.719680 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:30.719661 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 02:19:30.831912 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:30.831837 2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:19:31.471114 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.471075 2557 apiserver.go:52] "Watching apiserver" Jul 22 02:19:31.480490 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.480409 2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 02:19:31.482189 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.481733 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-rzszs","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal","openshift-multus/multus-7h5ws","openshift-multus/multus-additional-cni-plugins-m5st4","openshift-network-diagnostics/network-check-target-tqrxj","openshift-ovn-kubernetes/ovnkube-node-zf29q","kube-system/global-pull-secret-syncer-w4bgx","kube-system/konnectivity-agent-b2b7n","kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw","openshift-multus/network-metrics-daemon-mxnss","openshift-network-operator/iptables-alerter-rv4fz","openshift-cluster-node-tuning-operator/tuned-pcrqh","openshift-dns/node-resolver-wkh4j"] Jul 22 02:19:31.483360 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.483318 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:31.483470 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:31.483405 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:19:31.485623 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.485602 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.487093 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.487070 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.488530 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.488294 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.488530 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.488496 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 02:19:31.488646 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.488549 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:19:31.488684 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.488639 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p7f5f\"" Jul 22 02:19:31.490554 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.490077 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 02:19:31.490554 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.490413 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 02:19:31.490554 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.490532 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ddlq6\"" Jul 22 02:19:31.491153 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.491134 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 02:19:31.492082 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.492062 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 02:19:31.492082 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.491207 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 02:19:31.492245 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.491645 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 02:19:31.492245 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.491843 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-h2zsd\"" Jul 22 02:19:31.493929 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.492787 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:31.493929 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:31.492882 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:19:31.495327 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.495304 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.496669 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.496649 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wkh4j" Jul 22 02:19:31.497027 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.497010 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 02:19:31.497865 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.497844 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rzszs" Jul 22 02:19:31.498128 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.498105 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 02:19:31.498398 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.498377 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 02:19:31.499657 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.499640 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 02:19:31.499849 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.499827 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 02:19:31.499956 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.499829 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 02:19:31.499956 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.499922 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4fwmz\"" Jul 22 02:19:31.500064 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.499840 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 02:19:31.500064 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.500007 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qcdzd\"" Jul 22 02:19:31.500170 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.500105 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b2b7n" Jul 22 02:19:31.500170 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.500123 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 02:19:31.500276 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.500174 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 02:19:31.500631 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.500603 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 02:19:31.501049 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.501028 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ll872\"" Jul 22 02:19:31.501587 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.501039 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 02:19:31.501887 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.501087 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 02:19:31.502222 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.502001 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.503913 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.503383 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 02:19:31.503913 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.503402 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-b4x5k\"" Jul 22 02:19:31.503913 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.503661 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 02:19:31.504235 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.504214 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:31.504335 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:31.504288 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:19:31.504838 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.504818 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rv4fz" Jul 22 02:19:31.505657 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.505635 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 02:19:31.505908 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.505876 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dbj7m\"" Jul 22 02:19:31.506258 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.506237 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 02:19:31.506702 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.506678 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 02:19:31.507626 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.507604 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-hostroot\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.507738 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.507642 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-slash\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.507738 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.507669 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-kubernetes\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.507738 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.507701 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-sysctl-conf\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.507738 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.507725 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-etc-kubernetes\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.507974 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.507751 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-kubelet\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.507974 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.507777 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-run-ovn-kubernetes\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.507974 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.507804 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gv4m\" (UniqueName: \"kubernetes.io/projected/a16f14c0-d854-40e4-8151-abebbe9b1176-kube-api-access-6gv4m\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.507974 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.507829 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90aed964-e484-4a30-a581-fc933f03f780-tmp-dir\") pod \"node-resolver-wkh4j\" (UID: \"90aed964-e484-4a30-a581-fc933f03f780\") " pod="openshift-dns/node-resolver-wkh4j" Jul 22 02:19:31.507974 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.507855 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a16f14c0-d854-40e4-8151-abebbe9b1176-cnibin\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.507974 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.507925 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a16f14c0-d854-40e4-8151-abebbe9b1176-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.507974 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.507954 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfgz9\" (UniqueName: \"kubernetes.io/projected/f5a9c948-f382-4517-8029-e53165240e54-kube-api-access-cfgz9\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.508323 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.507990 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ede3658a-126b-446f-b57a-678aefbc7936-dbus\") pod \"global-pull-secret-syncer-w4bgx\" (UID: \"ede3658a-126b-446f-b57a-678aefbc7936\") " pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:31.508323 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508018 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 02:19:31.508323 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508028 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zd75h\"" Jul 22 02:19:31.508323 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508020 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/519f47d6-aef0-4eae-8eba-240fe5cadfed-serviceca\") pod \"node-ca-rzszs\" (UID: \"519f47d6-aef0-4eae-8eba-240fe5cadfed\") " pod="openshift-image-registry/node-ca-rzszs" Jul 22 02:19:31.508323 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508182 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:19:31.508323 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508227 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret\") pod \"global-pull-secret-syncer-w4bgx\" (UID: \"ede3658a-126b-446f-b57a-678aefbc7936\") " pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:31.508323 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508239 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 02:19:31.508323 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508256 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-var-lib-cni-bin\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.508323 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508284 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-var-lib-cni-multus\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.508323 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508326 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-cni-bin\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.508864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508351 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f5a9c948-f382-4517-8029-e53165240e54-ovnkube-script-lib\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.508864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508378 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-cni-netd\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.508864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508403 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-sysconfig\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.508864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508443 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-sysctl-d\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.508864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508460 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-run\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.508864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508482 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-multus-conf-dir\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.508864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508489 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 02:19:31.508864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508519 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlbp5\" (UniqueName: \"kubernetes.io/projected/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-kube-api-access-vlbp5\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.508864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508564 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a16f14c0-d854-40e4-8151-abebbe9b1176-system-cni-dir\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.508864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508585 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-run-netns\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.508864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508612 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-run-systemd\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.508864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508641 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a16f14c0-d854-40e4-8151-abebbe9b1176-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.508864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508661 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6gfw\" (UniqueName: \"kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw\") pod \"network-check-target-tqrxj\" (UID: \"d8fbc209-ac7d-480a-b8e2-3e4083f94ec9\") " pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:31.508864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508677 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/519f47d6-aef0-4eae-8eba-240fe5cadfed-host\") pod \"node-ca-rzszs\" (UID: \"519f47d6-aef0-4eae-8eba-240fe5cadfed\") " pod="openshift-image-registry/node-ca-rzszs" Jul 22 02:19:31.508864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508699 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-tuned\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.508864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508725 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bsm8\" (UniqueName: \"kubernetes.io/projected/90aed964-e484-4a30-a581-fc933f03f780-kube-api-access-4bsm8\") pod \"node-resolver-wkh4j\" (UID: \"90aed964-e484-4a30-a581-fc933f03f780\") " pod="openshift-dns/node-resolver-wkh4j" Jul 22 02:19:31.508864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508745 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-var-lib-openvswitch\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.509739 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508764 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-tmp\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.509739 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508783 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-system-cni-dir\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.509739 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508802 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-systemd\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.509739 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508818 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-lib-modules\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.509739 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508834 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-multus-cni-dir\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.509739 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508876 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-cni-binary-copy\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.509739 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508898 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-multus-daemon-config\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.509739 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508920 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-run-multus-certs\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.509739 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508943 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/90aed964-e484-4a30-a581-fc933f03f780-hosts-file\") pod \"node-resolver-wkh4j\" (UID: \"90aed964-e484-4a30-a581-fc933f03f780\") " pod="openshift-dns/node-resolver-wkh4j" Jul 22 02:19:31.509739 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508962 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-os-release\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.509739 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.508997 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-etc-openvswitch\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.509739 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509019 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f5a9c948-f382-4517-8029-e53165240e54-env-overrides\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.509739 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509049 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f5a9c948-f382-4517-8029-e53165240e54-ovn-node-metrics-cert\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.509739 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509077 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ede3658a-126b-446f-b57a-678aefbc7936-kubelet-config\") pod \"global-pull-secret-syncer-w4bgx\" (UID: \"ede3658a-126b-446f-b57a-678aefbc7936\") " pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:31.509739 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509097 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-host\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.509739 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509121 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz76d\" (UniqueName: \"kubernetes.io/projected/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-kube-api-access-lz76d\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.509739 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509143 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-var-lib-kubelet\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.510586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509159 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a16f14c0-d854-40e4-8151-abebbe9b1176-os-release\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.510586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509173 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-node-log\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.510586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509217 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-modprobe-d\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.510586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509253 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-cnibin\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.510586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509281 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-run-netns\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.510586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509306 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-systemd-units\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.510586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509335 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-run-openvswitch\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.510586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509361 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.510586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509386 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbx8v\" (UniqueName: \"kubernetes.io/projected/519f47d6-aef0-4eae-8eba-240fe5cadfed-kube-api-access-bbx8v\") pod \"node-ca-rzszs\" (UID: \"519f47d6-aef0-4eae-8eba-240fe5cadfed\") " pod="openshift-image-registry/node-ca-rzszs" Jul 22 02:19:31.510586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509410 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-multus-socket-dir-parent\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.510586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509434 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-run-k8s-cni-cncf-io\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.510586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509460 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a16f14c0-d854-40e4-8151-abebbe9b1176-cni-binary-copy\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.510586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509500 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-log-socket\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.510586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509525 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f5a9c948-f382-4517-8029-e53165240e54-ovnkube-config\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.510586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509549 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a16f14c0-d854-40e4-8151-abebbe9b1176-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.510586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509585 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-run-ovn\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.511458 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509608 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-sys\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.511458 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.509622 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-var-lib-kubelet\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.537917 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.537882 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2rv5q" Jul 22 02:19:31.546224 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.546192 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2rv5q" Jul 22 02:19:31.574412 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.571184 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:19:31.599297 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.599264 2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 02:19:31.610314 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610277 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" event={"ID":"11b830b0e1c88ebf098f99a124478494","Type":"ContainerStarted","Data":"9e6f657e6d3aca790ea8383d5f10954fe7799e4041bf4f1d54e76c5a14665d76"} Jul 22 02:19:31.610477 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610328 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-run-multus-certs\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.610477 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610363 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/90aed964-e484-4a30-a581-fc933f03f780-hosts-file\") pod \"node-resolver-wkh4j\" (UID: \"90aed964-e484-4a30-a581-fc933f03f780\") " pod="openshift-dns/node-resolver-wkh4j" Jul 22 02:19:31.610477 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610392 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e64b46d5-3ac5-43fd-8b38-253f8a8b04ad-konnectivity-ca\") pod \"konnectivity-agent-b2b7n\" (UID: \"e64b46d5-3ac5-43fd-8b38-253f8a8b04ad\") " pod="kube-system/konnectivity-agent-b2b7n" Jul 22 02:19:31.610477 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610420 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-os-release\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.610477 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610443 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-etc-openvswitch\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.610477 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610464 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f5a9c948-f382-4517-8029-e53165240e54-env-overrides\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.610863 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610488 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f5a9c948-f382-4517-8029-e53165240e54-ovn-node-metrics-cert\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.610863 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610508 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/90aed964-e484-4a30-a581-fc933f03f780-hosts-file\") pod \"node-resolver-wkh4j\" (UID: \"90aed964-e484-4a30-a581-fc933f03f780\") " pod="openshift-dns/node-resolver-wkh4j" Jul 22 02:19:31.610863 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610580 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-os-release\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.610863 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610584 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ede3658a-126b-446f-b57a-678aefbc7936-kubelet-config\") pod \"global-pull-secret-syncer-w4bgx\" (UID: \"ede3658a-126b-446f-b57a-678aefbc7936\") " pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:31.610863 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610512 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ede3658a-126b-446f-b57a-678aefbc7936-kubelet-config\") pod \"global-pull-secret-syncer-w4bgx\" (UID: \"ede3658a-126b-446f-b57a-678aefbc7936\") " pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:31.610863 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610664 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-etc-openvswitch\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.610863 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610696 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-host\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.610863 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610722 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz76d\" (UniqueName: \"kubernetes.io/projected/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-kube-api-access-lz76d\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.610863 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610750 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-var-lib-kubelet\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.610863 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610776 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a16f14c0-d854-40e4-8151-abebbe9b1176-os-release\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.610863 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610801 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-node-log\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.610863 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610828 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-modprobe-d\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.610863 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610853 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-cnibin\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.611550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610875 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-run-netns\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.611550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610926 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e64b46d5-3ac5-43fd-8b38-253f8a8b04ad-agent-certs\") pod \"konnectivity-agent-b2b7n\" (UID: \"e64b46d5-3ac5-43fd-8b38-253f8a8b04ad\") " pod="kube-system/konnectivity-agent-b2b7n" Jul 22 02:19:31.611550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610954 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-systemd-units\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.611550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.610979 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-run-openvswitch\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.611550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611006 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.611550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611033 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbx8v\" (UniqueName: \"kubernetes.io/projected/519f47d6-aef0-4eae-8eba-240fe5cadfed-kube-api-access-bbx8v\") pod \"node-ca-rzszs\" (UID: \"519f47d6-aef0-4eae-8eba-240fe5cadfed\") " pod="openshift-image-registry/node-ca-rzszs" Jul 22 02:19:31.611550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611043 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f5a9c948-f382-4517-8029-e53165240e54-env-overrides\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.611550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611063 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-multus-socket-dir-parent\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.611550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611091 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-run-k8s-cni-cncf-io\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.611550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611119 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nlqx\" (UniqueName: \"kubernetes.io/projected/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-kube-api-access-5nlqx\") pod \"network-metrics-daemon-mxnss\" (UID: \"72a0ac8c-4aaf-45c2-bd02-41b8e80e770b\") " pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:31.611550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611147 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a16f14c0-d854-40e4-8151-abebbe9b1176-cni-binary-copy\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.611550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611175 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-log-socket\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.611550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611201 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f5a9c948-f382-4517-8029-e53165240e54-ovnkube-config\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.611550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611239 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9cb72678-8713-456f-b26a-1919e879829e-iptables-alerter-script\") pod \"iptables-alerter-rv4fz\" (UID: \"9cb72678-8713-456f-b26a-1919e879829e\") " pod="openshift-network-operator/iptables-alerter-rv4fz" Jul 22 02:19:31.611550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611286 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a16f14c0-d854-40e4-8151-abebbe9b1176-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.611550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611321 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-run-ovn\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.611550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611346 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-sys\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.612478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611344 2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 02:19:31.612478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611388 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-run-openvswitch\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.612478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611416 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-var-lib-kubelet\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.612478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611434 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-hostroot\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.612478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611454 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-device-dir\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.612478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611472 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-slash\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.612478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611488 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-kubernetes\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.612478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611506 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-sysctl-conf\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.612478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611525 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-etc-kubernetes\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.612478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611543 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzhqx\" (UniqueName: \"kubernetes.io/projected/f6904572-ea00-4773-b4a4-c038962648a4-kube-api-access-bzhqx\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.612478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611563 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-kubelet\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.612478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611582 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-run-ovn-kubernetes\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.612478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611598 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-socket-dir\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.612478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611614 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-registration-dir\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.612478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611684 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gv4m\" (UniqueName: \"kubernetes.io/projected/a16f14c0-d854-40e4-8151-abebbe9b1176-kube-api-access-6gv4m\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.612478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611722 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90aed964-e484-4a30-a581-fc933f03f780-tmp-dir\") pod \"node-resolver-wkh4j\" (UID: \"90aed964-e484-4a30-a581-fc933f03f780\") " pod="openshift-dns/node-resolver-wkh4j" Jul 22 02:19:31.612478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611758 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-var-lib-kubelet\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611813 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-kubernetes\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611814 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-slash\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611850 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-run-multus-certs\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611930 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-host\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611987 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-var-lib-kubelet\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611995 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-etc-kubernetes\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612054 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-run-netns\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612074 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-cnibin\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612105 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-systemd-units\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612117 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-sysctl-conf\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.611766 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-hostroot\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612150 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-kubelet\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612177 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-run-ovn-kubernetes\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612219 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-node-log\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612272 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a16f14c0-d854-40e4-8151-abebbe9b1176-os-release\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612287 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a16f14c0-d854-40e4-8151-abebbe9b1176-cni-binary-copy\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612324 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-run-k8s-cni-cncf-io\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.613316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612340 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-etc-selinux\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.614263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612511 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.614263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612570 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-log-socket\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.614263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612594 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-multus-socket-dir-parent\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.614263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612673 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-modprobe-d\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.614263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612714 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a16f14c0-d854-40e4-8151-abebbe9b1176-cnibin\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.614263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612745 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a16f14c0-d854-40e4-8151-abebbe9b1176-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.614263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612775 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfgz9\" (UniqueName: \"kubernetes.io/projected/f5a9c948-f382-4517-8029-e53165240e54-kube-api-access-cfgz9\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.614263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612801 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90aed964-e484-4a30-a581-fc933f03f780-tmp-dir\") pod \"node-resolver-wkh4j\" (UID: \"90aed964-e484-4a30-a581-fc933f03f780\") " pod="openshift-dns/node-resolver-wkh4j" Jul 22 02:19:31.614263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612827 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-run-ovn\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.614263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612874 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a16f14c0-d854-40e4-8151-abebbe9b1176-cnibin\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.614263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612897 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ede3658a-126b-446f-b57a-678aefbc7936-dbus\") pod \"global-pull-secret-syncer-w4bgx\" (UID: \"ede3658a-126b-446f-b57a-678aefbc7936\") " pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:31.614263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612944 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f5a9c948-f382-4517-8029-e53165240e54-ovnkube-config\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.614263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.612829 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ede3658a-126b-446f-b57a-678aefbc7936-dbus\") pod \"global-pull-secret-syncer-w4bgx\" (UID: \"ede3658a-126b-446f-b57a-678aefbc7936\") " pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:31.614263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613012 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs\") pod \"network-metrics-daemon-mxnss\" (UID: \"72a0ac8c-4aaf-45c2-bd02-41b8e80e770b\") " pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:31.614263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613043 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.614263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613132 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9cb72678-8713-456f-b26a-1919e879829e-host-slash\") pod \"iptables-alerter-rv4fz\" (UID: \"9cb72678-8713-456f-b26a-1919e879829e\") " pod="openshift-network-operator/iptables-alerter-rv4fz" Jul 22 02:19:31.614263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613136 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-sys\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.615207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613167 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/519f47d6-aef0-4eae-8eba-240fe5cadfed-serviceca\") pod \"node-ca-rzszs\" (UID: \"519f47d6-aef0-4eae-8eba-240fe5cadfed\") " pod="openshift-image-registry/node-ca-rzszs" Jul 22 02:19:31.615207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613195 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret\") pod \"global-pull-secret-syncer-w4bgx\" (UID: \"ede3658a-126b-446f-b57a-678aefbc7936\") " pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:31.615207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613271 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-var-lib-cni-bin\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.615207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613289 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a16f14c0-d854-40e4-8151-abebbe9b1176-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.615207 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:31.613342 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:31.615207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613358 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-var-lib-cni-multus\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.615207 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:31.613415 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret podName:ede3658a-126b-446f-b57a-678aefbc7936 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:32.113392067 +0000 UTC m=+3.022257064 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret") pod "global-pull-secret-syncer-w4bgx" (UID: "ede3658a-126b-446f-b57a-678aefbc7936") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:31.615207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613437 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-var-lib-cni-multus\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.615207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613477 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-cni-bin\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.615207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613439 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-cni-bin\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.615207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613513 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-host-var-lib-cni-bin\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.615207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613551 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f5a9c948-f382-4517-8029-e53165240e54-ovnkube-script-lib\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.615207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613595 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-cni-netd\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.615207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613617 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-sysconfig\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.615207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613633 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-sysctl-d\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.615207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613633 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a16f14c0-d854-40e4-8151-abebbe9b1176-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.615207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613657 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/519f47d6-aef0-4eae-8eba-240fe5cadfed-serviceca\") pod \"node-ca-rzszs\" (UID: \"519f47d6-aef0-4eae-8eba-240fe5cadfed\") " pod="openshift-image-registry/node-ca-rzszs" Jul 22 02:19:31.615883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613668 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-run\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.615883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613702 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-cni-netd\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.615883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613709 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-multus-conf-dir\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.615883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613723 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-run\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.615883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613738 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlbp5\" (UniqueName: \"kubernetes.io/projected/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-kube-api-access-vlbp5\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.615883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613759 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-sysctl-d\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.615883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613765 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-sysconfig\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.615883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613790 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcctl\" (UniqueName: \"kubernetes.io/projected/9cb72678-8713-456f-b26a-1919e879829e-kube-api-access-mcctl\") pod \"iptables-alerter-rv4fz\" (UID: \"9cb72678-8713-456f-b26a-1919e879829e\") " pod="openshift-network-operator/iptables-alerter-rv4fz" Jul 22 02:19:31.615883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613778 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-multus-conf-dir\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.615883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613819 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a16f14c0-d854-40e4-8151-abebbe9b1176-system-cni-dir\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.615883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613847 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a16f14c0-d854-40e4-8151-abebbe9b1176-system-cni-dir\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.615883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613876 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-run-netns\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.615883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613917 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-run-systemd\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.615883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613913 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-host-run-netns\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.615883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613945 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a16f14c0-d854-40e4-8151-abebbe9b1176-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.615883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613974 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6gfw\" (UniqueName: \"kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw\") pod \"network-check-target-tqrxj\" (UID: \"d8fbc209-ac7d-480a-b8e2-3e4083f94ec9\") " pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:31.615883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613984 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-run-systemd\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.613994 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/519f47d6-aef0-4eae-8eba-240fe5cadfed-host\") pod \"node-ca-rzszs\" (UID: \"519f47d6-aef0-4eae-8eba-240fe5cadfed\") " pod="openshift-image-registry/node-ca-rzszs" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614042 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/519f47d6-aef0-4eae-8eba-240fe5cadfed-host\") pod \"node-ca-rzszs\" (UID: \"519f47d6-aef0-4eae-8eba-240fe5cadfed\") " pod="openshift-image-registry/node-ca-rzszs" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614065 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-tuned\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614060 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f5a9c948-f382-4517-8029-e53165240e54-ovnkube-script-lib\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614088 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bsm8\" (UniqueName: \"kubernetes.io/projected/90aed964-e484-4a30-a581-fc933f03f780-kube-api-access-4bsm8\") pod \"node-resolver-wkh4j\" (UID: \"90aed964-e484-4a30-a581-fc933f03f780\") " pod="openshift-dns/node-resolver-wkh4j" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614110 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-var-lib-openvswitch\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614133 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-tmp\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614137 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a16f14c0-d854-40e4-8151-abebbe9b1176-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614166 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-system-cni-dir\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614192 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a9c948-f382-4517-8029-e53165240e54-var-lib-openvswitch\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614196 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-sys-fs\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614243 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-systemd\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614249 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-system-cni-dir\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614268 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-lib-modules\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614293 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-multus-cni-dir\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614311 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-systemd\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614318 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-cni-binary-copy\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.616498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614352 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-multus-daemon-config\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.617152 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614426 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-lib-modules\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.617152 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614493 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-multus-cni-dir\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.617152 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.614825 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-cni-binary-copy\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.617152 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.615283 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-multus-daemon-config\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.617152 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.615691 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f5a9c948-f382-4517-8029-e53165240e54-ovn-node-metrics-cert\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.617458 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.617433 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-etc-tuned\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.617732 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.617712 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-tmp\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.623862 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:31.622224 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:19:31.623862 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:31.622251 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:19:31.623862 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:31.622264 2557 projected.go:196] Error preparing data for projected volume kube-api-access-k6gfw for pod openshift-network-diagnostics/network-check-target-tqrxj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:31.623862 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:31.622325 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw podName:d8fbc209-ac7d-480a-b8e2-3e4083f94ec9 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:32.122305588 +0000 UTC m=+3.031170566 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k6gfw" (UniqueName: "kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw") pod "network-check-target-tqrxj" (UID: "d8fbc209-ac7d-480a-b8e2-3e4083f94ec9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:31.623862 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.623087 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz76d\" (UniqueName: \"kubernetes.io/projected/0abe5f9a-00db-47d5-8f0e-894e8fc71de3-kube-api-access-lz76d\") pod \"tuned-pcrqh\" (UID: \"0abe5f9a-00db-47d5-8f0e-894e8fc71de3\") " pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.623862 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.623239 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gv4m\" (UniqueName: \"kubernetes.io/projected/a16f14c0-d854-40e4-8151-abebbe9b1176-kube-api-access-6gv4m\") pod \"multus-additional-cni-plugins-m5st4\" (UID: \"a16f14c0-d854-40e4-8151-abebbe9b1176\") " pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.623862 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.623448 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbx8v\" (UniqueName: \"kubernetes.io/projected/519f47d6-aef0-4eae-8eba-240fe5cadfed-kube-api-access-bbx8v\") pod \"node-ca-rzszs\" (UID: \"519f47d6-aef0-4eae-8eba-240fe5cadfed\") " pod="openshift-image-registry/node-ca-rzszs" Jul 22 02:19:31.625045 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.624739 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfgz9\" (UniqueName: \"kubernetes.io/projected/f5a9c948-f382-4517-8029-e53165240e54-kube-api-access-cfgz9\") pod \"ovnkube-node-zf29q\" (UID: \"f5a9c948-f382-4517-8029-e53165240e54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.625559 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.625536 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlbp5\" (UniqueName: \"kubernetes.io/projected/ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1-kube-api-access-vlbp5\") pod \"multus-7h5ws\" (UID: \"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1\") " pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.626731 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.626705 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bsm8\" (UniqueName: \"kubernetes.io/projected/90aed964-e484-4a30-a581-fc933f03f780-kube-api-access-4bsm8\") pod \"node-resolver-wkh4j\" (UID: \"90aed964-e484-4a30-a581-fc933f03f780\") " pod="openshift-dns/node-resolver-wkh4j" Jul 22 02:19:31.714983 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.714941 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e64b46d5-3ac5-43fd-8b38-253f8a8b04ad-agent-certs\") pod \"konnectivity-agent-b2b7n\" (UID: \"e64b46d5-3ac5-43fd-8b38-253f8a8b04ad\") " pod="kube-system/konnectivity-agent-b2b7n" Jul 22 02:19:31.714983 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.714987 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nlqx\" (UniqueName: \"kubernetes.io/projected/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-kube-api-access-5nlqx\") pod \"network-metrics-daemon-mxnss\" (UID: \"72a0ac8c-4aaf-45c2-bd02-41b8e80e770b\") " pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:31.715242 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.715016 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9cb72678-8713-456f-b26a-1919e879829e-iptables-alerter-script\") pod \"iptables-alerter-rv4fz\" (UID: \"9cb72678-8713-456f-b26a-1919e879829e\") " pod="openshift-network-operator/iptables-alerter-rv4fz" Jul 22 02:19:31.715242 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.715048 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-device-dir\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.715242 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.715130 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-device-dir\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.715480 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.715448 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzhqx\" (UniqueName: \"kubernetes.io/projected/f6904572-ea00-4773-b4a4-c038962648a4-kube-api-access-bzhqx\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.715561 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.715505 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-socket-dir\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.715561 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.715531 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-registration-dir\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.715673 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.715564 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-etc-selinux\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.715673 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.715595 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs\") pod \"network-metrics-daemon-mxnss\" (UID: \"72a0ac8c-4aaf-45c2-bd02-41b8e80e770b\") " pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:31.715673 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.715621 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.715673 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.715654 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9cb72678-8713-456f-b26a-1919e879829e-host-slash\") pod \"iptables-alerter-rv4fz\" (UID: \"9cb72678-8713-456f-b26a-1919e879829e\") " pod="openshift-network-operator/iptables-alerter-rv4fz" Jul 22 02:19:31.715913 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.715716 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcctl\" (UniqueName: \"kubernetes.io/projected/9cb72678-8713-456f-b26a-1919e879829e-kube-api-access-mcctl\") pod \"iptables-alerter-rv4fz\" (UID: \"9cb72678-8713-456f-b26a-1919e879829e\") " pod="openshift-network-operator/iptables-alerter-rv4fz" Jul 22 02:19:31.715913 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.715764 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-sys-fs\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.715913 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.715797 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e64b46d5-3ac5-43fd-8b38-253f8a8b04ad-konnectivity-ca\") pod \"konnectivity-agent-b2b7n\" (UID: \"e64b46d5-3ac5-43fd-8b38-253f8a8b04ad\") " pod="kube-system/konnectivity-agent-b2b7n" Jul 22 02:19:31.715913 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:31.715892 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:31.716128 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.715937 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9cb72678-8713-456f-b26a-1919e879829e-iptables-alerter-script\") pod \"iptables-alerter-rv4fz\" (UID: \"9cb72678-8713-456f-b26a-1919e879829e\") " pod="openshift-network-operator/iptables-alerter-rv4fz" Jul 22 02:19:31.716128 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:31.715973 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs podName:72a0ac8c-4aaf-45c2-bd02-41b8e80e770b nodeName:}" failed. No retries permitted until 2026-07-22 02:19:32.215952009 +0000 UTC m=+3.124816998 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs") pod "network-metrics-daemon-mxnss" (UID: "72a0ac8c-4aaf-45c2-bd02-41b8e80e770b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:31.716128 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.716021 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9cb72678-8713-456f-b26a-1919e879829e-host-slash\") pod \"iptables-alerter-rv4fz\" (UID: \"9cb72678-8713-456f-b26a-1919e879829e\") " pod="openshift-network-operator/iptables-alerter-rv4fz" Jul 22 02:19:31.716128 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.716042 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-registration-dir\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.716128 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.716059 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.716412 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.716156 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-etc-selinux\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.716412 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.716155 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-socket-dir\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.716412 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.716262 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e64b46d5-3ac5-43fd-8b38-253f8a8b04ad-konnectivity-ca\") pod \"konnectivity-agent-b2b7n\" (UID: \"e64b46d5-3ac5-43fd-8b38-253f8a8b04ad\") " pod="kube-system/konnectivity-agent-b2b7n" Jul 22 02:19:31.716412 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.716297 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f6904572-ea00-4773-b4a4-c038962648a4-sys-fs\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.718385 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.718358 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e64b46d5-3ac5-43fd-8b38-253f8a8b04ad-agent-certs\") pod \"konnectivity-agent-b2b7n\" (UID: \"e64b46d5-3ac5-43fd-8b38-253f8a8b04ad\") " pod="kube-system/konnectivity-agent-b2b7n" Jul 22 02:19:31.725173 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.725136 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nlqx\" (UniqueName: \"kubernetes.io/projected/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-kube-api-access-5nlqx\") pod \"network-metrics-daemon-mxnss\" (UID: \"72a0ac8c-4aaf-45c2-bd02-41b8e80e770b\") " pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:31.725429 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.725396 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzhqx\" (UniqueName: \"kubernetes.io/projected/f6904572-ea00-4773-b4a4-c038962648a4-kube-api-access-bzhqx\") pod \"aws-ebs-csi-driver-node-qwbxw\" (UID: \"f6904572-ea00-4773-b4a4-c038962648a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.728098 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.728074 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcctl\" (UniqueName: \"kubernetes.io/projected/9cb72678-8713-456f-b26a-1919e879829e-kube-api-access-mcctl\") pod \"iptables-alerter-rv4fz\" (UID: \"9cb72678-8713-456f-b26a-1919e879829e\") " pod="openshift-network-operator/iptables-alerter-rv4fz" Jul 22 02:19:31.801242 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.801155 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" Jul 22 02:19:31.810651 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.810626 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7h5ws" Jul 22 02:19:31.822150 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.822117 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m5st4" Jul 22 02:19:31.828166 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.828142 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:31.837067 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.837044 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wkh4j" Jul 22 02:19:31.845914 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.845891 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rzszs" Jul 22 02:19:31.854694 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.854672 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b2b7n" Jul 22 02:19:31.862710 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.862687 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" Jul 22 02:19:31.872625 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:31.872604 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rv4fz" Jul 22 02:19:32.119483 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:32.119391 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret\") pod \"global-pull-secret-syncer-w4bgx\" (UID: \"ede3658a-126b-446f-b57a-678aefbc7936\") " pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:32.119646 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:32.119577 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:32.119691 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:32.119683 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret podName:ede3658a-126b-446f-b57a-678aefbc7936 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:33.119659072 +0000 UTC m=+4.028524053 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret") pod "global-pull-secret-syncer-w4bgx" (UID: "ede3658a-126b-446f-b57a-678aefbc7936") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:32.220207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:32.220171 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6gfw\" (UniqueName: \"kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw\") pod \"network-check-target-tqrxj\" (UID: \"d8fbc209-ac7d-480a-b8e2-3e4083f94ec9\") " pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:32.220387 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:32.220240 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs\") pod \"network-metrics-daemon-mxnss\" (UID: \"72a0ac8c-4aaf-45c2-bd02-41b8e80e770b\") " pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:32.220387 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:32.220354 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:32.220387 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:32.220357 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:19:32.220387 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:32.220385 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:19:32.220579 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:32.220398 2557 projected.go:196] Error preparing data for projected volume kube-api-access-k6gfw for pod openshift-network-diagnostics/network-check-target-tqrxj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:32.220579 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:32.220423 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs podName:72a0ac8c-4aaf-45c2-bd02-41b8e80e770b nodeName:}" failed. No retries permitted until 2026-07-22 02:19:33.220405442 +0000 UTC m=+4.129270417 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs") pod "network-metrics-daemon-mxnss" (UID: "72a0ac8c-4aaf-45c2-bd02-41b8e80e770b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:32.220579 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:32.220456 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw podName:d8fbc209-ac7d-480a-b8e2-3e4083f94ec9 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:33.220438479 +0000 UTC m=+4.129303454 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6gfw" (UniqueName: "kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw") pod "network-check-target-tqrxj" (UID: "d8fbc209-ac7d-480a-b8e2-3e4083f94ec9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:32.548972 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:32.548823 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 02:14:32 +0000 UTC" deadline="2027-12-17 13:49:07.832029412 +0000 UTC" Jul 22 02:19:32.548972 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:32.548911 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12323h29m35.283123317s" Jul 22 02:19:32.603445 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:32.603406 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:32.603641 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:32.603545 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:19:32.603641 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:32.603602 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:32.603753 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:32.603715 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:19:32.629947 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:32.629913 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:19:32.687541 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:32.687517 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6904572_ea00_4773_b4a4_c038962648a4.slice/crio-afde0170d0dfed72f18d649ea5daf1a3790f25f7c66d819b29851af8a9c339a4 WatchSource:0}: Error finding container afde0170d0dfed72f18d649ea5daf1a3790f25f7c66d819b29851af8a9c339a4: Status 404 returned error can't find the container with id afde0170d0dfed72f18d649ea5daf1a3790f25f7c66d819b29851af8a9c339a4 Jul 22 02:19:32.691437 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:32.689397 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90aed964_e484_4a30_a581_fc933f03f780.slice/crio-b6500ee100bef480a8db64f33a5e80eb0ace2931a4ac69c25a23036e1f66df65 WatchSource:0}: Error finding container b6500ee100bef480a8db64f33a5e80eb0ace2931a4ac69c25a23036e1f66df65: Status 404 returned error can't find the container with id b6500ee100bef480a8db64f33a5e80eb0ace2931a4ac69c25a23036e1f66df65 Jul 22 02:19:32.691437 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:32.690253 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce0f9c32_6e0b_4f17_a4a6_59aea33b8ab1.slice/crio-740ed6aeba4e7d9ebec3c221dc4b60eed7b878994300d2c8ef11ddcd8880c67e WatchSource:0}: Error finding container 740ed6aeba4e7d9ebec3c221dc4b60eed7b878994300d2c8ef11ddcd8880c67e: Status 404 returned error can't find the container with id 740ed6aeba4e7d9ebec3c221dc4b60eed7b878994300d2c8ef11ddcd8880c67e Jul 22 02:19:32.692037 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:32.691500 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda16f14c0_d854_40e4_8151_abebbe9b1176.slice/crio-c4103aa9338f2c4c11698bd27ad2001162182a223afdd85154aeba6207f72469 WatchSource:0}: Error finding container c4103aa9338f2c4c11698bd27ad2001162182a223afdd85154aeba6207f72469: Status 404 returned error can't find the container with id c4103aa9338f2c4c11698bd27ad2001162182a223afdd85154aeba6207f72469 Jul 22 02:19:32.694470 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:32.694453 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb72678_8713_456f_b26a_1919e879829e.slice/crio-e695c138398d10f98e078b7641b5da89d86efd72af832d2565593f8d52cb0e06 WatchSource:0}: Error finding container e695c138398d10f98e078b7641b5da89d86efd72af832d2565593f8d52cb0e06: Status 404 returned error can't find the container with id e695c138398d10f98e078b7641b5da89d86efd72af832d2565593f8d52cb0e06 Jul 22 02:19:32.695415 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:32.695392 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode64b46d5_3ac5_43fd_8b38_253f8a8b04ad.slice/crio-b822f6eb4384e763a445ed167c6ad0ed5b46129449e907189e9bcb9efd0c0aa7 WatchSource:0}: Error finding container b822f6eb4384e763a445ed167c6ad0ed5b46129449e907189e9bcb9efd0c0aa7: Status 404 returned error can't find the container with id b822f6eb4384e763a445ed167c6ad0ed5b46129449e907189e9bcb9efd0c0aa7 Jul 22 02:19:32.697676 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:32.697045 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5a9c948_f382_4517_8029_e53165240e54.slice/crio-9fdc183c4f1662f7b080e9762c30f387af05eaa6ba3f55cde248fd5555983e47 WatchSource:0}: Error finding container 9fdc183c4f1662f7b080e9762c30f387af05eaa6ba3f55cde248fd5555983e47: Status 404 returned error can't find the container with id 9fdc183c4f1662f7b080e9762c30f387af05eaa6ba3f55cde248fd5555983e47 Jul 22 02:19:32.700323 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:19:32.700300 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod519f47d6_aef0_4eae_8eba_240fe5cadfed.slice/crio-bfbb6d206a6c1ecc34db8b02bc19ce64050b88aa4880f55c0445aa282ecf153e WatchSource:0}: Error finding container bfbb6d206a6c1ecc34db8b02bc19ce64050b88aa4880f55c0445aa282ecf153e: Status 404 returned error can't find the container with id bfbb6d206a6c1ecc34db8b02bc19ce64050b88aa4880f55c0445aa282ecf153e Jul 22 02:19:33.126429 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.126190 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret\") pod \"global-pull-secret-syncer-w4bgx\" (UID: \"ede3658a-126b-446f-b57a-678aefbc7936\") " pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:33.126617 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:33.126456 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:33.126617 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:33.126549 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret podName:ede3658a-126b-446f-b57a-678aefbc7936 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:35.126525316 +0000 UTC m=+6.035390295 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret") pod "global-pull-secret-syncer-w4bgx" (UID: "ede3658a-126b-446f-b57a-678aefbc7936") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:33.227271 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.227231 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs\") pod \"network-metrics-daemon-mxnss\" (UID: \"72a0ac8c-4aaf-45c2-bd02-41b8e80e770b\") " pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:33.227460 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.227306 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6gfw\" (UniqueName: \"kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw\") pod \"network-check-target-tqrxj\" (UID: \"d8fbc209-ac7d-480a-b8e2-3e4083f94ec9\") " pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:33.227523 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:33.227458 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:19:33.227523 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:33.227476 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:19:33.227523 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:33.227493 2557 projected.go:196] Error preparing data for projected volume kube-api-access-k6gfw for pod openshift-network-diagnostics/network-check-target-tqrxj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:33.227716 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:33.227554 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw podName:d8fbc209-ac7d-480a-b8e2-3e4083f94ec9 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:35.227535127 +0000 UTC m=+6.136400108 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6gfw" (UniqueName: "kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw") pod "network-check-target-tqrxj" (UID: "d8fbc209-ac7d-480a-b8e2-3e4083f94ec9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:33.228014 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:33.227991 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:33.228139 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:33.228059 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs podName:72a0ac8c-4aaf-45c2-bd02-41b8e80e770b nodeName:}" failed. No retries permitted until 2026-07-22 02:19:35.228042807 +0000 UTC m=+6.136907788 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs") pod "network-metrics-daemon-mxnss" (UID: "72a0ac8c-4aaf-45c2-bd02-41b8e80e770b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:33.549313 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.549265 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 02:14:32 +0000 UTC" deadline="2027-12-27 05:56:43.530860605 +0000 UTC" Jul 22 02:19:33.549313 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.549310 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12555h37m9.981555306s" Jul 22 02:19:33.606413 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.606372 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:33.606585 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:33.606536 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:19:33.615642 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.615522 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" event={"ID":"f5a9c948-f382-4517-8029-e53165240e54","Type":"ContainerStarted","Data":"9fdc183c4f1662f7b080e9762c30f387af05eaa6ba3f55cde248fd5555983e47"} Jul 22 02:19:33.618277 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.618204 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b2b7n" event={"ID":"e64b46d5-3ac5-43fd-8b38-253f8a8b04ad","Type":"ContainerStarted","Data":"b822f6eb4384e763a445ed167c6ad0ed5b46129449e907189e9bcb9efd0c0aa7"} Jul 22 02:19:33.620018 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.619946 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rv4fz" event={"ID":"9cb72678-8713-456f-b26a-1919e879829e","Type":"ContainerStarted","Data":"e695c138398d10f98e078b7641b5da89d86efd72af832d2565593f8d52cb0e06"} Jul 22 02:19:33.622672 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.622623 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7h5ws" event={"ID":"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1","Type":"ContainerStarted","Data":"740ed6aeba4e7d9ebec3c221dc4b60eed7b878994300d2c8ef11ddcd8880c67e"} Jul 22 02:19:33.639662 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.639573 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rzszs" event={"ID":"519f47d6-aef0-4eae-8eba-240fe5cadfed","Type":"ContainerStarted","Data":"bfbb6d206a6c1ecc34db8b02bc19ce64050b88aa4880f55c0445aa282ecf153e"} Jul 22 02:19:33.647060 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.647023 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m5st4" event={"ID":"a16f14c0-d854-40e4-8151-abebbe9b1176","Type":"ContainerStarted","Data":"c4103aa9338f2c4c11698bd27ad2001162182a223afdd85154aeba6207f72469"} Jul 22 02:19:33.654753 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.654723 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" event={"ID":"f6904572-ea00-4773-b4a4-c038962648a4","Type":"ContainerStarted","Data":"afde0170d0dfed72f18d649ea5daf1a3790f25f7c66d819b29851af8a9c339a4"} Jul 22 02:19:33.666495 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.666463 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" event={"ID":"0abe5f9a-00db-47d5-8f0e-894e8fc71de3","Type":"ContainerStarted","Data":"5f0b376177c70e09c302b5b910ceb4744b70c2e628efb05baed561555906db66"} Jul 22 02:19:33.685203 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.685166 2557 generic.go:356] "Generic (PLEG): container finished" podID="47803356514fd5d371d5e4ac55377679" containerID="14fe2bb95a0cbbc00b122952397d48b509869b308d8cfdf6cf4869d13e537480" exitCode=0 Jul 22 02:19:33.685567 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.685486 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" event={"ID":"47803356514fd5d371d5e4ac55377679","Type":"ContainerDied","Data":"14fe2bb95a0cbbc00b122952397d48b509869b308d8cfdf6cf4869d13e537480"} Jul 22 02:19:33.693826 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.693795 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" event={"ID":"11b830b0e1c88ebf098f99a124478494","Type":"ContainerStarted","Data":"007d83bbfc1cf38e56dd802d3ef692890c23692ef3536d8cc1eae1b6309f3504"} Jul 22 02:19:33.720055 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.720004 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wkh4j" event={"ID":"90aed964-e484-4a30-a581-fc933f03f780","Type":"ContainerStarted","Data":"b6500ee100bef480a8db64f33a5e80eb0ace2931a4ac69c25a23036e1f66df65"} Jul 22 02:19:33.723070 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:33.723010 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" podStartSLOduration=2.72299047 podStartE2EDuration="2.72299047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:33.722161146 +0000 UTC m=+4.631026147" watchObservedRunningTime="2026-07-22 02:19:33.72299047 +0000 UTC m=+4.631855472" Jul 22 02:19:34.603337 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:34.603302 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:34.603833 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:34.603302 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:34.603833 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:34.603489 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:19:34.603833 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:34.603597 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:19:34.750950 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:34.750864 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" event={"ID":"47803356514fd5d371d5e4ac55377679","Type":"ContainerStarted","Data":"a3b501825d3e38df5572f9637f494527b76cfa58bfa2356307634177192c3aac"} Jul 22 02:19:34.769984 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:34.768270 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" podStartSLOduration=3.768247567 podStartE2EDuration="3.768247567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:34.768127444 +0000 UTC m=+5.676992445" watchObservedRunningTime="2026-07-22 02:19:34.768247567 +0000 UTC m=+5.677112564" Jul 22 02:19:35.146056 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:35.146010 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret\") pod \"global-pull-secret-syncer-w4bgx\" (UID: \"ede3658a-126b-446f-b57a-678aefbc7936\") " pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:35.146263 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:35.146169 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:35.146263 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:35.146237 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret podName:ede3658a-126b-446f-b57a-678aefbc7936 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:39.146216791 +0000 UTC m=+10.055081772 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret") pod "global-pull-secret-syncer-w4bgx" (UID: "ede3658a-126b-446f-b57a-678aefbc7936") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:35.246451 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:35.246406 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6gfw\" (UniqueName: \"kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw\") pod \"network-check-target-tqrxj\" (UID: \"d8fbc209-ac7d-480a-b8e2-3e4083f94ec9\") " pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:35.246659 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:35.246511 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs\") pod \"network-metrics-daemon-mxnss\" (UID: \"72a0ac8c-4aaf-45c2-bd02-41b8e80e770b\") " pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:35.246728 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:35.246667 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:35.246785 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:35.246732 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs podName:72a0ac8c-4aaf-45c2-bd02-41b8e80e770b nodeName:}" failed. No retries permitted until 2026-07-22 02:19:39.246712914 +0000 UTC m=+10.155577894 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs") pod "network-metrics-daemon-mxnss" (UID: "72a0ac8c-4aaf-45c2-bd02-41b8e80e770b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:35.247231 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:35.247209 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:19:35.247317 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:35.247238 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:19:35.247317 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:35.247251 2557 projected.go:196] Error preparing data for projected volume kube-api-access-k6gfw for pod openshift-network-diagnostics/network-check-target-tqrxj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:35.247317 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:35.247302 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw podName:d8fbc209-ac7d-480a-b8e2-3e4083f94ec9 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:39.247286532 +0000 UTC m=+10.156151512 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6gfw" (UniqueName: "kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw") pod "network-check-target-tqrxj" (UID: "d8fbc209-ac7d-480a-b8e2-3e4083f94ec9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:35.604716 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:35.604159 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:35.604716 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:35.604309 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:19:36.603617 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:36.603586 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:36.603870 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:36.603729 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:19:36.603870 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:36.603809 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:36.604001 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:36.603944 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:19:37.604546 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:37.603986 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:37.604546 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:37.604136 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:19:38.603387 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:38.603307 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:38.603526 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:38.603432 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:19:38.603841 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:38.603308 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:38.603954 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:38.603907 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:19:39.183977 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:39.183901 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret\") pod \"global-pull-secret-syncer-w4bgx\" (UID: \"ede3658a-126b-446f-b57a-678aefbc7936\") " pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:39.184422 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:39.184048 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:39.184422 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:39.184108 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret podName:ede3658a-126b-446f-b57a-678aefbc7936 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:47.184089921 +0000 UTC m=+18.092954894 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret") pod "global-pull-secret-syncer-w4bgx" (UID: "ede3658a-126b-446f-b57a-678aefbc7936") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:39.285056 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:39.284311 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs\") pod \"network-metrics-daemon-mxnss\" (UID: \"72a0ac8c-4aaf-45c2-bd02-41b8e80e770b\") " pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:39.285056 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:39.284375 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6gfw\" (UniqueName: \"kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw\") pod \"network-check-target-tqrxj\" (UID: \"d8fbc209-ac7d-480a-b8e2-3e4083f94ec9\") " pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:39.285056 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:39.284493 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:19:39.285056 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:39.284511 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:19:39.285056 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:39.284522 2557 projected.go:196] Error preparing data for projected volume kube-api-access-k6gfw for pod openshift-network-diagnostics/network-check-target-tqrxj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:39.285056 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:39.284572 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw podName:d8fbc209-ac7d-480a-b8e2-3e4083f94ec9 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:47.284556237 +0000 UTC m=+18.193421211 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6gfw" (UniqueName: "kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw") pod "network-check-target-tqrxj" (UID: "d8fbc209-ac7d-480a-b8e2-3e4083f94ec9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:39.285056 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:39.284975 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:39.285056 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:39.285029 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs podName:72a0ac8c-4aaf-45c2-bd02-41b8e80e770b nodeName:}" failed. No retries permitted until 2026-07-22 02:19:47.285012159 +0000 UTC m=+18.193877137 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs") pod "network-metrics-daemon-mxnss" (UID: "72a0ac8c-4aaf-45c2-bd02-41b8e80e770b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:39.605009 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:39.604589 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:39.605009 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:39.604709 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:19:40.603535 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:40.603497 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:40.603986 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:40.603544 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:40.603986 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:40.603640 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:19:40.603986 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:40.603759 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:19:41.604123 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:41.604093 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:41.604565 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:41.604243 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:19:42.603064 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:42.603023 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:42.603261 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:42.603159 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:19:42.603555 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:42.603534 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:42.603706 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:42.603632 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:19:43.603361 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:43.603332 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:43.603849 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:43.603476 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:19:44.602962 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:44.602927 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:44.602962 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:44.602963 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:44.603159 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:44.603042 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:19:44.603225 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:44.603188 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:19:45.603269 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:45.603233 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:45.603740 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:45.603376 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:19:46.603883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:46.603842 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:46.604949 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:46.604034 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:19:46.604949 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:46.604592 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:46.604949 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:46.604738 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:19:47.243515 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:47.243480 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret\") pod \"global-pull-secret-syncer-w4bgx\" (UID: \"ede3658a-126b-446f-b57a-678aefbc7936\") " pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:47.243687 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:47.243625 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:47.243741 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:47.243687 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret podName:ede3658a-126b-446f-b57a-678aefbc7936 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:03.243667428 +0000 UTC m=+34.152532403 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret") pod "global-pull-secret-syncer-w4bgx" (UID: "ede3658a-126b-446f-b57a-678aefbc7936") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:47.344213 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:47.344174 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs\") pod \"network-metrics-daemon-mxnss\" (UID: \"72a0ac8c-4aaf-45c2-bd02-41b8e80e770b\") " pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:47.344389 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:47.344231 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6gfw\" (UniqueName: \"kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw\") pod \"network-check-target-tqrxj\" (UID: \"d8fbc209-ac7d-480a-b8e2-3e4083f94ec9\") " pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:47.344389 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:47.344334 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:47.344389 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:47.344358 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:19:47.344389 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:47.344378 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:19:47.344389 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:47.344390 2557 projected.go:196] Error preparing data for projected volume kube-api-access-k6gfw for pod openshift-network-diagnostics/network-check-target-tqrxj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:47.344614 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:47.344410 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs podName:72a0ac8c-4aaf-45c2-bd02-41b8e80e770b nodeName:}" failed. No retries permitted until 2026-07-22 02:20:03.344390063 +0000 UTC m=+34.253255058 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs") pod "network-metrics-daemon-mxnss" (UID: "72a0ac8c-4aaf-45c2-bd02-41b8e80e770b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:47.344614 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:47.344437 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw podName:d8fbc209-ac7d-480a-b8e2-3e4083f94ec9 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:03.344424096 +0000 UTC m=+34.253289086 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6gfw" (UniqueName: "kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw") pod "network-check-target-tqrxj" (UID: "d8fbc209-ac7d-480a-b8e2-3e4083f94ec9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:47.603438 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:47.603356 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:47.603601 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:47.603485 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:19:48.603065 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:48.603029 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:48.603539 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:48.603077 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:48.603539 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:48.603158 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:19:48.603539 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:48.603285 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:19:49.603554 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:49.603526 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:49.604060 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:49.603616 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:19:50.603259 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:50.603224 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:50.603415 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:50.603330 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:19:50.603415 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:50.603387 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:50.603493 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:50.603481 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:19:51.603022 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:51.602983 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:51.603466 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:51.603122 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:19:52.603661 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.603245 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:52.604387 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.603245 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:52.604387 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:52.603789 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:19:52.604387 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:52.603864 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:19:52.790187 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.790159 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7h5ws" event={"ID":"ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1","Type":"ContainerStarted","Data":"051d2bece05e5eab77b5c1a09ad3afdf4d7eddcedfe9b30de5b78a83e509eea9"} Jul 22 02:19:52.791515 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.791490 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rzszs" event={"ID":"519f47d6-aef0-4eae-8eba-240fe5cadfed","Type":"ContainerStarted","Data":"b0ce20cbb102ffd2b1b17fed929a5044daec203b383de7c612d48f22aae62b0a"} Jul 22 02:19:52.792973 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.792948 2557 generic.go:356] "Generic (PLEG): container finished" podID="a16f14c0-d854-40e4-8151-abebbe9b1176" containerID="3a2735791c66f178be288e40d2b67f216c43aa439c1321d9a602211b0f031c48" exitCode=0 Jul 22 02:19:52.793071 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.793034 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m5st4" event={"ID":"a16f14c0-d854-40e4-8151-abebbe9b1176","Type":"ContainerDied","Data":"3a2735791c66f178be288e40d2b67f216c43aa439c1321d9a602211b0f031c48"} Jul 22 02:19:52.795025 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.794999 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" event={"ID":"f6904572-ea00-4773-b4a4-c038962648a4","Type":"ContainerStarted","Data":"633a26daa518c07088b5314f2097561c097455310dea16b21ac1ee62c38a655e"} Jul 22 02:19:52.796465 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.796442 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" event={"ID":"0abe5f9a-00db-47d5-8f0e-894e8fc71de3","Type":"ContainerStarted","Data":"23ea5ad8a38543f42a01da429cc1783f55940f22056585aa407af95ad1adeed3"} Jul 22 02:19:52.798072 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.798051 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wkh4j" event={"ID":"90aed964-e484-4a30-a581-fc933f03f780","Type":"ContainerStarted","Data":"8746c5e5c887da9cabf91c68216db7bf0de691a003e1bd0876a78f76036b4952"} Jul 22 02:19:52.800584 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.800566 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" event={"ID":"f5a9c948-f382-4517-8029-e53165240e54","Type":"ContainerStarted","Data":"c69a17c03a07b53b5b29d738fe4a620ed70f02824b385fb57a0f64933040bf42"} Jul 22 02:19:52.800665 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.800622 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" event={"ID":"f5a9c948-f382-4517-8029-e53165240e54","Type":"ContainerStarted","Data":"6513891fd6523eec3ec755fc374f4b6dc11aeb2e7b27744d7c7465b2e67f6117"} Jul 22 02:19:52.800716 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.800669 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" event={"ID":"f5a9c948-f382-4517-8029-e53165240e54","Type":"ContainerStarted","Data":"0a01c4003ad8f48b5dbe27cf16989948a43abab18565f520770464762715f186"} Jul 22 02:19:52.800716 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.800700 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" event={"ID":"f5a9c948-f382-4517-8029-e53165240e54","Type":"ContainerStarted","Data":"7d2142d7f20fab9570ffb17164618a52434dac9446de23f92ad6f954732ca318"} Jul 22 02:19:52.801773 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.801750 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b2b7n" event={"ID":"e64b46d5-3ac5-43fd-8b38-253f8a8b04ad","Type":"ContainerStarted","Data":"9c1f896b0985ab76b7d17765dfe82f540f7ebfc0d78467402d1466b67827b6b5"} Jul 22 02:19:52.811049 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.811016 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7h5ws" podStartSLOduration=3.177605112 podStartE2EDuration="22.811005124s" totalImagesPullingTime="19.633400012s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:52.810726263 +0000 UTC m=+23.719591258" watchObservedRunningTime="2026-07-22 02:19:52.811005124 +0000 UTC m=+23.719870119" Jul 22 02:19:52.852017 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.851979 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pcrqh" podStartSLOduration=3.260421899 podStartE2EDuration="22.851963663s" totalImagesPullingTime="19.591541764s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:52.828465345 +0000 UTC m=+23.737330342" watchObservedRunningTime="2026-07-22 02:19:52.851963663 +0000 UTC m=+23.760828658" Jul 22 02:19:52.865979 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.865941 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wkh4j" podStartSLOduration=3.272256051 podStartE2EDuration="22.865926234s" totalImagesPullingTime="19.593670183s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:52.865617543 +0000 UTC m=+23.774482538" watchObservedRunningTime="2026-07-22 02:19:52.865926234 +0000 UTC m=+23.774791229" Jul 22 02:19:52.881621 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.881560 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rzszs" podStartSLOduration=3.302792173 podStartE2EDuration="22.881542574s" totalImagesPullingTime="19.578750401s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:52.880931567 +0000 UTC m=+23.789796575" watchObservedRunningTime="2026-07-22 02:19:52.881542574 +0000 UTC m=+23.790407570" Jul 22 02:19:52.902528 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:52.902489 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-b2b7n" podStartSLOduration=3.32605572 podStartE2EDuration="22.902475871s" totalImagesPullingTime="19.576420151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:52.902340737 +0000 UTC m=+23.811205736" watchObservedRunningTime="2026-07-22 02:19:52.902475871 +0000 UTC m=+23.811340865" Jul 22 02:19:53.193928 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:53.193850 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-b2b7n" Jul 22 02:19:53.194631 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:53.194606 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-b2b7n" Jul 22 02:19:53.603533 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:53.603075 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:53.603533 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:53.603231 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:19:53.675609 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:53.675584 2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 02:19:53.807588 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:53.807495 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" event={"ID":"f5a9c948-f382-4517-8029-e53165240e54","Type":"ContainerStarted","Data":"41284681f6d8c08da5f1a9ee02ede7cad13d64efe6cb42b88429b8c6844d7d10"} Jul 22 02:19:53.807806 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:53.807622 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" event={"ID":"f5a9c948-f382-4517-8029-e53165240e54","Type":"ContainerStarted","Data":"149cd98f68adad445b637922f7faa9bd68fe2fc5e8f67745392131acf13c7c7b"} Jul 22 02:19:53.809231 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:53.809203 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rv4fz" event={"ID":"9cb72678-8713-456f-b26a-1919e879829e","Type":"ContainerStarted","Data":"797b4a671163e7cffa89250e4e103eec4193c29f6649825fa9a9c18200f0a939"} Jul 22 02:19:53.811423 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:53.811397 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" event={"ID":"f6904572-ea00-4773-b4a4-c038962648a4","Type":"ContainerStarted","Data":"6671233291f29cdee79ef54c557aae8b00163ec80e178eb6292cb6ef68a9ed75"} Jul 22 02:19:53.811749 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:53.811726 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b2b7n" Jul 22 02:19:53.812220 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:53.812200 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b2b7n" Jul 22 02:19:53.826342 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:53.826291 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rv4fz" podStartSLOduration=4.24438836 podStartE2EDuration="23.826276598s" totalImagesPullingTime="19.581888238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:53.825838561 +0000 UTC m=+24.734703549" watchObservedRunningTime="2026-07-22 02:19:53.826276598 +0000 UTC m=+24.735141593" Jul 22 02:19:54.585040 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:54.584896 2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T02:19:53.675604343Z","UUID":"d371994b-ed00-47d2-b2c5-5202ff3c4350","Handler":null,"Name":"","Endpoint":""} Jul 22 02:19:54.586736 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:54.586712 2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 02:19:54.586911 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:54.586759 2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 02:19:54.603734 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:54.603696 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:54.604174 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:54.603710 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:54.604174 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:54.603813 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:19:54.604174 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:54.603886 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:19:55.274571 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:55.274490 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2b7n_e64b46d5-3ac5-43fd-8b38-253f8a8b04ad/konnectivity-agent/0.log" limit=1024 Jul 22 02:19:55.355968 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:55.355937 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2b7n_e64b46d5-3ac5-43fd-8b38-253f8a8b04ad/konnectivity-agent/0.log" limit=1024 Jul 22 02:19:55.603605 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:55.603525 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:55.603764 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:55.603637 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:19:55.819130 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:55.819091 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" event={"ID":"f5a9c948-f382-4517-8029-e53165240e54","Type":"ContainerStarted","Data":"a4ed7b3c973986d70235b831c762e7725d4fe66e8dc50ec6b5b550369d9a9ee0"} Jul 22 02:19:55.821174 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:55.821142 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" event={"ID":"f6904572-ea00-4773-b4a4-c038962648a4","Type":"ContainerStarted","Data":"664a666de14c6d7a22c11ea7e9c48c817308bbb06d1ad6ae800403296b7079d1"} Jul 22 02:19:55.843305 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:55.841641 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwbxw" podStartSLOduration=4.155618552 podStartE2EDuration="25.841623626s" totalImagesPullingTime="21.686005074s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:55.840505245 +0000 UTC m=+26.749370242" watchObservedRunningTime="2026-07-22 02:19:55.841623626 +0000 UTC m=+26.750488622" Jul 22 02:19:56.604053 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:56.603806 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:56.604407 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:56.603868 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:56.604407 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:56.604154 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:19:56.604407 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:56.604203 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:19:57.172786 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:57.172741 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2b7n_e64b46d5-3ac5-43fd-8b38-253f8a8b04ad/konnectivity-agent/0.log" limit=1024 Jul 22 02:19:57.272351 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:57.272313 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2b7n_e64b46d5-3ac5-43fd-8b38-253f8a8b04ad/konnectivity-agent/0.log" limit=1024 Jul 22 02:19:57.603282 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:57.603248 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:57.603475 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:57.603437 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:19:58.405220 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:58.405193 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2b7n_e64b46d5-3ac5-43fd-8b38-253f8a8b04ad/konnectivity-agent/0.log" limit=1024 Jul 22 02:19:58.475155 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:58.475123 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2b7n_e64b46d5-3ac5-43fd-8b38-253f8a8b04ad/konnectivity-agent/0.log" limit=1024 Jul 22 02:19:58.603455 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:58.603392 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:19:58.603583 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:58.603392 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:19:58.603583 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:58.603482 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:19:58.603677 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:58.603581 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:19:58.830082 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:58.830050 2557 generic.go:356] "Generic (PLEG): container finished" podID="a16f14c0-d854-40e4-8151-abebbe9b1176" containerID="a8467b348425ee9d9ede23686b276cd16be31b8fc761c2bcc8f70991c69a8777" exitCode=0 Jul 22 02:19:58.830249 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:58.830139 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m5st4" event={"ID":"a16f14c0-d854-40e4-8151-abebbe9b1176","Type":"ContainerDied","Data":"a8467b348425ee9d9ede23686b276cd16be31b8fc761c2bcc8f70991c69a8777"} Jul 22 02:19:58.833760 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:58.833742 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" event={"ID":"f5a9c948-f382-4517-8029-e53165240e54","Type":"ContainerStarted","Data":"f7809c0f76b9c6e371229fd0d7cc976295dc5037b42d341edb5d1fc13925ceef"} Jul 22 02:19:58.834175 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:58.834032 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:58.834175 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:58.834105 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:58.834175 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:58.834137 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:58.850089 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:58.850065 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:58.850191 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:58.850175 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:19:58.880700 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:58.880624 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" podStartSLOduration=9.24141594 podStartE2EDuration="28.880609929s" totalImagesPullingTime="19.639193989s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:58.878975769 +0000 UTC m=+29.787840764" watchObservedRunningTime="2026-07-22 02:19:58.880609929 +0000 UTC m=+29.789474948" Jul 22 02:19:59.603650 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:19:59.603489 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:19:59.604245 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:19:59.603723 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:20:00.121419 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:00.121327 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2b7n_e64b46d5-3ac5-43fd-8b38-253f8a8b04ad/konnectivity-agent/0.log" limit=1024 Jul 22 02:20:00.179046 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:00.179013 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2b7n_e64b46d5-3ac5-43fd-8b38-253f8a8b04ad/konnectivity-agent/0.log" limit=1024 Jul 22 02:20:00.369213 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:00.369179 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mxnss"] Jul 22 02:20:00.369401 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:00.369295 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:20:00.369459 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:00.369401 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:20:00.372108 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:00.372033 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w4bgx"] Jul 22 02:20:00.372229 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:00.372155 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:20:00.372288 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:00.372259 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:20:00.372810 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:00.372787 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tqrxj"] Jul 22 02:20:00.372949 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:00.372886 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:20:00.373006 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:00.372984 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:20:00.840035 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:00.840001 2557 generic.go:356] "Generic (PLEG): container finished" podID="a16f14c0-d854-40e4-8151-abebbe9b1176" containerID="cdfcb7a68f2e5dc3daa28208a4c58bcf42ce7057db98bd5c4294118977f3a45e" exitCode=0 Jul 22 02:20:00.840397 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:00.840090 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m5st4" event={"ID":"a16f14c0-d854-40e4-8151-abebbe9b1176","Type":"ContainerDied","Data":"cdfcb7a68f2e5dc3daa28208a4c58bcf42ce7057db98bd5c4294118977f3a45e"} Jul 22 02:20:01.603413 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:01.603378 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:20:01.603413 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:01.603413 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:20:01.603651 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:01.603531 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:20:01.603651 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:01.603612 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:20:02.603343 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:02.603307 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:20:02.603713 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:02.603416 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:20:02.848464 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:02.848431 2557 generic.go:356] "Generic (PLEG): container finished" podID="a16f14c0-d854-40e4-8151-abebbe9b1176" containerID="4eeb87422ef61751d3f1b7b691fbcfb2618b5f13127e3221d2ca1470731ff809" exitCode=0 Jul 22 02:20:02.848616 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:02.848492 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m5st4" event={"ID":"a16f14c0-d854-40e4-8151-abebbe9b1176","Type":"ContainerDied","Data":"4eeb87422ef61751d3f1b7b691fbcfb2618b5f13127e3221d2ca1470731ff809"} Jul 22 02:20:03.261759 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:03.261730 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret\") pod \"global-pull-secret-syncer-w4bgx\" (UID: \"ede3658a-126b-446f-b57a-678aefbc7936\") " pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:20:03.261963 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:03.261872 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:20:03.261963 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:03.261930 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret podName:ede3658a-126b-446f-b57a-678aefbc7936 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:35.261916898 +0000 UTC m=+66.170781871 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret") pod "global-pull-secret-syncer-w4bgx" (UID: "ede3658a-126b-446f-b57a-678aefbc7936") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:20:03.363046 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:03.363011 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6gfw\" (UniqueName: \"kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw\") pod \"network-check-target-tqrxj\" (UID: \"d8fbc209-ac7d-480a-b8e2-3e4083f94ec9\") " pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:20:03.363235 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:03.363072 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs\") pod \"network-metrics-daemon-mxnss\" (UID: \"72a0ac8c-4aaf-45c2-bd02-41b8e80e770b\") " pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:20:03.363235 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:03.363192 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:20:03.363235 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:03.363208 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:20:03.363235 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:03.363235 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:20:03.363428 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:03.363248 2557 projected.go:196] Error preparing data for projected volume kube-api-access-k6gfw for pod openshift-network-diagnostics/network-check-target-tqrxj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:20:03.363428 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:03.363262 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs podName:72a0ac8c-4aaf-45c2-bd02-41b8e80e770b nodeName:}" failed. No retries permitted until 2026-07-22 02:20:35.363243267 +0000 UTC m=+66.272108243 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs") pod "network-metrics-daemon-mxnss" (UID: "72a0ac8c-4aaf-45c2-bd02-41b8e80e770b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:20:03.363428 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:03.363298 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw podName:d8fbc209-ac7d-480a-b8e2-3e4083f94ec9 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:35.363281828 +0000 UTC m=+66.272146860 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6gfw" (UniqueName: "kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw") pod "network-check-target-tqrxj" (UID: "d8fbc209-ac7d-480a-b8e2-3e4083f94ec9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:20:03.603445 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:03.603358 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:20:03.603894 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:03.603478 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:20:03.603894 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:03.603519 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:20:03.603894 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:03.603576 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:20:04.603622 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:04.603586 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:20:04.604058 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:04.603702 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w4bgx" podUID="ede3658a-126b-446f-b57a-678aefbc7936" Jul 22 02:20:05.603621 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.603590 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:20:05.603807 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.603590 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:20:05.603807 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:05.603716 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqrxj" podUID="d8fbc209-ac7d-480a-b8e2-3e4083f94ec9" Jul 22 02:20:05.603807 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:05.603781 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mxnss" podUID="72a0ac8c-4aaf-45c2-bd02-41b8e80e770b" Jul 22 02:20:05.913547 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.913468 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeReady" Jul 22 02:20:05.913694 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.913629 2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 02:20:05.948627 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.948596 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84c6f8455b-2hn2c"] Jul 22 02:20:05.951302 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.951278 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:05.954228 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.954204 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f4l4l\"" Jul 22 02:20:05.954228 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.954211 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 02:20:05.954228 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.954236 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 02:20:05.954466 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.954208 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 02:20:05.960595 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.959787 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 02:20:05.960692 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.960675 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zgnxf"] Jul 22 02:20:05.963358 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.963335 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mwwl8"] Jul 22 02:20:05.964657 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.964640 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:05.965620 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.965589 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mwwl8" Jul 22 02:20:05.967851 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.967252 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4k4rg\"" Jul 22 02:20:05.967851 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.967349 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 02:20:05.967851 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.967548 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 02:20:05.967851 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.967551 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84c6f8455b-2hn2c"] Jul 22 02:20:05.968108 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.967979 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 02:20:05.968258 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.968235 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 02:20:05.968258 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.968245 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ptnrq\"" Jul 22 02:20:05.968470 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.968276 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 02:20:05.975294 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.975272 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zgnxf"] Jul 22 02:20:05.975392 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:05.975328 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mwwl8"] Jul 22 02:20:06.086227 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.086190 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6e35b92-2a47-4594-8b5e-9e8110892637-trusted-ca\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.086421 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.086236 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1a22a50d-1436-409d-af35-9dc1907dd74e-cert\") pod \"ingress-canary-mwwl8\" (UID: \"1a22a50d-1436-409d-af35-9dc1907dd74e\") " pod="openshift-ingress-canary/ingress-canary-mwwl8" Jul 22 02:20:06.086421 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.086260 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-config-volume\") pod \"dns-default-zgnxf\" (UID: \"c3fb8521-f63a-4773-ad73-73ae63e8b4d2\") " pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:06.086421 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.086283 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbg22\" (UniqueName: \"kubernetes.io/projected/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-kube-api-access-qbg22\") pod \"dns-default-zgnxf\" (UID: \"c3fb8521-f63a-4773-ad73-73ae63e8b4d2\") " pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:06.086421 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.086304 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6e35b92-2a47-4594-8b5e-9e8110892637-installation-pull-secrets\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.086421 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.086329 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6e35b92-2a47-4594-8b5e-9e8110892637-ca-trust-extracted\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.086421 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.086359 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6e35b92-2a47-4594-8b5e-9e8110892637-image-registry-private-configuration\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.086421 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.086414 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-bound-sa-token\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.086717 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.086485 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-certificates\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.086717 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.086515 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz5tr\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-kube-api-access-pz5tr\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.086717 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.086567 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pww8\" (UniqueName: \"kubernetes.io/projected/1a22a50d-1436-409d-af35-9dc1907dd74e-kube-api-access-4pww8\") pod \"ingress-canary-mwwl8\" (UID: \"1a22a50d-1436-409d-af35-9dc1907dd74e\") " pod="openshift-ingress-canary/ingress-canary-mwwl8" Jul 22 02:20:06.086717 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.086616 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.086717 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.086665 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls\") pod \"dns-default-zgnxf\" (UID: \"c3fb8521-f63a-4773-ad73-73ae63e8b4d2\") " pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:06.086717 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.086688 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-tmp-dir\") pod \"dns-default-zgnxf\" (UID: \"c3fb8521-f63a-4773-ad73-73ae63e8b4d2\") " pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:06.187667 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.187587 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-certificates\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.187667 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.187623 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz5tr\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-kube-api-access-pz5tr\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.187667 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.187659 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pww8\" (UniqueName: \"kubernetes.io/projected/1a22a50d-1436-409d-af35-9dc1907dd74e-kube-api-access-4pww8\") pod \"ingress-canary-mwwl8\" (UID: \"1a22a50d-1436-409d-af35-9dc1907dd74e\") " pod="openshift-ingress-canary/ingress-canary-mwwl8" Jul 22 02:20:06.187956 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.187849 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.187956 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.187909 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls\") pod \"dns-default-zgnxf\" (UID: \"c3fb8521-f63a-4773-ad73-73ae63e8b4d2\") " pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:06.187956 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.187932 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-tmp-dir\") pod \"dns-default-zgnxf\" (UID: \"c3fb8521-f63a-4773-ad73-73ae63e8b4d2\") " pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:06.187956 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.187952 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6e35b92-2a47-4594-8b5e-9e8110892637-trusted-ca\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.188169 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.187980 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1a22a50d-1436-409d-af35-9dc1907dd74e-cert\") pod \"ingress-canary-mwwl8\" (UID: \"1a22a50d-1436-409d-af35-9dc1907dd74e\") " pod="openshift-ingress-canary/ingress-canary-mwwl8" Jul 22 02:20:06.188169 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.188002 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-config-volume\") pod \"dns-default-zgnxf\" (UID: \"c3fb8521-f63a-4773-ad73-73ae63e8b4d2\") " pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:06.188169 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:06.188007 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:20:06.188169 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:06.188026 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c6f8455b-2hn2c: secret "image-registry-tls" not found Jul 22 02:20:06.188169 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:06.188054 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:20:06.188169 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.188025 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbg22\" (UniqueName: \"kubernetes.io/projected/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-kube-api-access-qbg22\") pod \"dns-default-zgnxf\" (UID: \"c3fb8521-f63a-4773-ad73-73ae63e8b4d2\") " pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:06.188169 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:06.188099 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls podName:b6e35b92-2a47-4594-8b5e-9e8110892637 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:06.688077053 +0000 UTC m=+37.596942026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls") pod "image-registry-84c6f8455b-2hn2c" (UID: "b6e35b92-2a47-4594-8b5e-9e8110892637") : secret "image-registry-tls" not found Jul 22 02:20:06.188418 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.188306 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-tmp-dir\") pod \"dns-default-zgnxf\" (UID: \"c3fb8521-f63a-4773-ad73-73ae63e8b4d2\") " pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:06.188418 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:06.188343 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls podName:c3fb8521-f63a-4773-ad73-73ae63e8b4d2 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:06.688326767 +0000 UTC m=+37.597191740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls") pod "dns-default-zgnxf" (UID: "c3fb8521-f63a-4773-ad73-73ae63e8b4d2") : secret "dns-default-metrics-tls" not found Jul 22 02:20:06.188418 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.188363 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6e35b92-2a47-4594-8b5e-9e8110892637-installation-pull-secrets\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.188418 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.188400 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6e35b92-2a47-4594-8b5e-9e8110892637-ca-trust-extracted\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.188593 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.188456 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6e35b92-2a47-4594-8b5e-9e8110892637-image-registry-private-configuration\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.188593 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.188486 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-bound-sa-token\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.188693 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.188610 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-config-volume\") pod \"dns-default-zgnxf\" (UID: \"c3fb8521-f63a-4773-ad73-73ae63e8b4d2\") " pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:06.188896 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.188870 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-certificates\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.189029 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.188976 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6e35b92-2a47-4594-8b5e-9e8110892637-ca-trust-extracted\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.189276 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.189251 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6e35b92-2a47-4594-8b5e-9e8110892637-trusted-ca\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.193315 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.193290 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6e35b92-2a47-4594-8b5e-9e8110892637-installation-pull-secrets\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.193315 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.193303 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6e35b92-2a47-4594-8b5e-9e8110892637-image-registry-private-configuration\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.193449 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.193298 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1a22a50d-1436-409d-af35-9dc1907dd74e-cert\") pod \"ingress-canary-mwwl8\" (UID: \"1a22a50d-1436-409d-af35-9dc1907dd74e\") " pod="openshift-ingress-canary/ingress-canary-mwwl8" Jul 22 02:20:06.204993 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.204961 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbg22\" (UniqueName: \"kubernetes.io/projected/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-kube-api-access-qbg22\") pod \"dns-default-zgnxf\" (UID: \"c3fb8521-f63a-4773-ad73-73ae63e8b4d2\") " pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:06.205184 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.205167 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-bound-sa-token\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.205266 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.205240 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz5tr\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-kube-api-access-pz5tr\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.205580 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.205476 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pww8\" (UniqueName: \"kubernetes.io/projected/1a22a50d-1436-409d-af35-9dc1907dd74e-kube-api-access-4pww8\") pod \"ingress-canary-mwwl8\" (UID: \"1a22a50d-1436-409d-af35-9dc1907dd74e\") " pod="openshift-ingress-canary/ingress-canary-mwwl8" Jul 22 02:20:06.283052 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.283022 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mwwl8" Jul 22 02:20:06.603208 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.603174 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:20:06.606549 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.606529 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 02:20:06.692078 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.692011 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls\") pod \"dns-default-zgnxf\" (UID: \"c3fb8521-f63a-4773-ad73-73ae63e8b4d2\") " pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:06.692279 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.692123 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:06.692279 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:06.692169 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:20:06.692279 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:06.692233 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls podName:c3fb8521-f63a-4773-ad73-73ae63e8b4d2 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:07.692212462 +0000 UTC m=+38.601077448 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls") pod "dns-default-zgnxf" (UID: "c3fb8521-f63a-4773-ad73-73ae63e8b4d2") : secret "dns-default-metrics-tls" not found Jul 22 02:20:06.692279 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:06.692239 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:20:06.692279 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:06.692251 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c6f8455b-2hn2c: secret "image-registry-tls" not found Jul 22 02:20:06.692502 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:06.692302 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls podName:b6e35b92-2a47-4594-8b5e-9e8110892637 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:07.692286337 +0000 UTC m=+38.601151315 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls") pod "image-registry-84c6f8455b-2hn2c" (UID: "b6e35b92-2a47-4594-8b5e-9e8110892637") : secret "image-registry-tls" not found Jul 22 02:20:06.819389 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.819360 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gmw9c"] Jul 22 02:20:06.825140 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.825103 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:06.827797 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.827743 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 02:20:06.827975 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.827791 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 02:20:06.828060 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.827986 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 02:20:06.828060 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.828020 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 02:20:06.828259 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.828240 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6sgmx\"" Jul 22 02:20:06.831982 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.831942 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gmw9c"] Jul 22 02:20:06.860975 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.860950 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mwwl8"] Jul 22 02:20:06.864228 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:20:06.864199 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a22a50d_1436_409d_af35_9dc1907dd74e.slice/crio-1b4b37f8ca2f17c16d6dbf8ee519096d0b84cc4ba49924514fce0f8f143cb606 WatchSource:0}: Error finding container 1b4b37f8ca2f17c16d6dbf8ee519096d0b84cc4ba49924514fce0f8f143cb606: Status 404 returned error can't find the container with id 1b4b37f8ca2f17c16d6dbf8ee519096d0b84cc4ba49924514fce0f8f143cb606 Jul 22 02:20:06.994966 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.994774 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlx9b\" (UniqueName: \"kubernetes.io/projected/13baf0c2-a61b-4602-970d-6bde2895bc5e-kube-api-access-tlx9b\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:06.995118 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.994980 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/13baf0c2-a61b-4602-970d-6bde2895bc5e-data-volume\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:06.995118 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.995012 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/13baf0c2-a61b-4602-970d-6bde2895bc5e-crio-socket\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:06.995118 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.995067 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/13baf0c2-a61b-4602-970d-6bde2895bc5e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:06.995221 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:06.995144 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:07.096344 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.096310 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/13baf0c2-a61b-4602-970d-6bde2895bc5e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:07.096344 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.096347 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:07.096583 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:07.096455 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:20:07.096583 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:07.096463 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:20:07.096583 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:07.096535 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/13baf0c2-a61b-4602-970d-6bde2895bc5e-kube-rbac-proxy-cm podName:13baf0c2-a61b-4602-970d-6bde2895bc5e nodeName:}" failed. No retries permitted until 2026-07-22 02:20:07.596515546 +0000 UTC m=+38.505380533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/13baf0c2-a61b-4602-970d-6bde2895bc5e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gmw9c" (UID: "13baf0c2-a61b-4602-970d-6bde2895bc5e") : configmap "kube-rbac-proxy" not found Jul 22 02:20:07.096583 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.096559 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlx9b\" (UniqueName: \"kubernetes.io/projected/13baf0c2-a61b-4602-970d-6bde2895bc5e-kube-api-access-tlx9b\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:07.096583 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:07.096575 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls podName:13baf0c2-a61b-4602-970d-6bde2895bc5e nodeName:}" failed. No retries permitted until 2026-07-22 02:20:07.596559228 +0000 UTC m=+38.505424202 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gmw9c" (UID: "13baf0c2-a61b-4602-970d-6bde2895bc5e") : secret "insights-runtime-extractor-tls" not found Jul 22 02:20:07.096799 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.096608 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/13baf0c2-a61b-4602-970d-6bde2895bc5e-data-volume\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:07.096799 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.096631 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/13baf0c2-a61b-4602-970d-6bde2895bc5e-crio-socket\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:07.096890 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.096871 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/13baf0c2-a61b-4602-970d-6bde2895bc5e-crio-socket\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:07.096924 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.096914 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/13baf0c2-a61b-4602-970d-6bde2895bc5e-data-volume\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:07.109708 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.109631 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlx9b\" (UniqueName: \"kubernetes.io/projected/13baf0c2-a61b-4602-970d-6bde2895bc5e-kube-api-access-tlx9b\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:07.600954 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.600911 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/13baf0c2-a61b-4602-970d-6bde2895bc5e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:07.601130 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.600967 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:07.601191 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:07.601163 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:20:07.601253 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:07.601240 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls podName:13baf0c2-a61b-4602-970d-6bde2895bc5e nodeName:}" failed. No retries permitted until 2026-07-22 02:20:08.601219612 +0000 UTC m=+39.510084642 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gmw9c" (UID: "13baf0c2-a61b-4602-970d-6bde2895bc5e") : secret "insights-runtime-extractor-tls" not found Jul 22 02:20:07.601663 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:07.601643 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:20:07.601740 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:07.601709 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/13baf0c2-a61b-4602-970d-6bde2895bc5e-kube-rbac-proxy-cm podName:13baf0c2-a61b-4602-970d-6bde2895bc5e nodeName:}" failed. No retries permitted until 2026-07-22 02:20:08.60168544 +0000 UTC m=+39.510550417 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/13baf0c2-a61b-4602-970d-6bde2895bc5e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gmw9c" (UID: "13baf0c2-a61b-4602-970d-6bde2895bc5e") : configmap "kube-rbac-proxy" not found Jul 22 02:20:07.604846 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.604766 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:20:07.605267 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.605056 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:20:07.607988 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.607962 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 02:20:07.608579 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.608186 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t8kv7\"" Jul 22 02:20:07.609068 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.609046 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 02:20:07.609172 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.609078 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8wzn5\"" Jul 22 02:20:07.609233 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.609046 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 02:20:07.702554 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.702522 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:07.702759 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.702573 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls\") pod \"dns-default-zgnxf\" (UID: \"c3fb8521-f63a-4773-ad73-73ae63e8b4d2\") " pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:07.702759 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:07.702719 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:20:07.702870 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:07.702791 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls podName:c3fb8521-f63a-4773-ad73-73ae63e8b4d2 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:09.702768647 +0000 UTC m=+40.611633635 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls") pod "dns-default-zgnxf" (UID: "c3fb8521-f63a-4773-ad73-73ae63e8b4d2") : secret "dns-default-metrics-tls" not found Jul 22 02:20:07.702870 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:07.702724 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:20:07.702870 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:07.702814 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c6f8455b-2hn2c: secret "image-registry-tls" not found Jul 22 02:20:07.702992 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:07.702882 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls podName:b6e35b92-2a47-4594-8b5e-9e8110892637 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:09.702869055 +0000 UTC m=+40.611734027 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls") pod "image-registry-84c6f8455b-2hn2c" (UID: "b6e35b92-2a47-4594-8b5e-9e8110892637") : secret "image-registry-tls" not found Jul 22 02:20:07.862382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.862304 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mwwl8" event={"ID":"1a22a50d-1436-409d-af35-9dc1907dd74e","Type":"ContainerStarted","Data":"1b4b37f8ca2f17c16d6dbf8ee519096d0b84cc4ba49924514fce0f8f143cb606"} Jul 22 02:20:07.866235 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.866202 2557 generic.go:356] "Generic (PLEG): container finished" podID="a16f14c0-d854-40e4-8151-abebbe9b1176" containerID="c4826e4f27aaf39044f57845878f2202f8735e50cce34ec60394874faf22569a" exitCode=0 Jul 22 02:20:07.866363 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:07.866265 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m5st4" event={"ID":"a16f14c0-d854-40e4-8151-abebbe9b1176","Type":"ContainerDied","Data":"c4826e4f27aaf39044f57845878f2202f8735e50cce34ec60394874faf22569a"} Jul 22 02:20:08.255726 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:08.255697 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-grvw8"] Jul 22 02:20:08.261870 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:08.261853 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-grvw8" Jul 22 02:20:08.265160 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:08.265137 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 22 02:20:08.265308 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:08.265245 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 22 02:20:08.265396 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:08.265376 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ss56s\"" Jul 22 02:20:08.272408 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:08.272384 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-grvw8"] Jul 22 02:20:08.408370 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:08.408293 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc77n\" (UniqueName: \"kubernetes.io/projected/25bca1ce-620d-4f9b-8f60-279b6e21668e-kube-api-access-nc77n\") pod \"migrator-7cc9c7ffd7-grvw8\" (UID: \"25bca1ce-620d-4f9b-8f60-279b6e21668e\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-grvw8" Jul 22 02:20:08.509212 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:08.509128 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc77n\" (UniqueName: \"kubernetes.io/projected/25bca1ce-620d-4f9b-8f60-279b6e21668e-kube-api-access-nc77n\") pod \"migrator-7cc9c7ffd7-grvw8\" (UID: \"25bca1ce-620d-4f9b-8f60-279b6e21668e\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-grvw8" Jul 22 02:20:08.522945 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:08.522917 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc77n\" (UniqueName: \"kubernetes.io/projected/25bca1ce-620d-4f9b-8f60-279b6e21668e-kube-api-access-nc77n\") pod \"migrator-7cc9c7ffd7-grvw8\" (UID: \"25bca1ce-620d-4f9b-8f60-279b6e21668e\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-grvw8" Jul 22 02:20:08.570313 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:08.570280 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-grvw8" Jul 22 02:20:08.609585 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:08.609546 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/13baf0c2-a61b-4602-970d-6bde2895bc5e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:08.609983 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:08.609598 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:08.609983 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:08.609670 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:20:08.609983 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:08.609745 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:20:08.609983 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:08.609757 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/13baf0c2-a61b-4602-970d-6bde2895bc5e-kube-rbac-proxy-cm podName:13baf0c2-a61b-4602-970d-6bde2895bc5e nodeName:}" failed. No retries permitted until 2026-07-22 02:20:10.609737025 +0000 UTC m=+41.518602012 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/13baf0c2-a61b-4602-970d-6bde2895bc5e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gmw9c" (UID: "13baf0c2-a61b-4602-970d-6bde2895bc5e") : configmap "kube-rbac-proxy" not found Jul 22 02:20:08.609983 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:08.609796 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls podName:13baf0c2-a61b-4602-970d-6bde2895bc5e nodeName:}" failed. No retries permitted until 2026-07-22 02:20:10.609780703 +0000 UTC m=+41.518645678 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gmw9c" (UID: "13baf0c2-a61b-4602-970d-6bde2895bc5e") : secret "insights-runtime-extractor-tls" not found Jul 22 02:20:08.615308 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:08.615281 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wkh4j_90aed964-e484-4a30-a581-fc933f03f780/dns-node-resolver/0.log" Jul 22 02:20:08.872677 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:08.872583 2557 generic.go:356] "Generic (PLEG): container finished" podID="a16f14c0-d854-40e4-8151-abebbe9b1176" containerID="ec811c8bc2177959fffab1251f75060d4780e02a81d84c499e807df11ab17800" exitCode=0 Jul 22 02:20:08.872677 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:08.872645 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m5st4" event={"ID":"a16f14c0-d854-40e4-8151-abebbe9b1176","Type":"ContainerDied","Data":"ec811c8bc2177959fffab1251f75060d4780e02a81d84c499e807df11ab17800"} Jul 22 02:20:09.330048 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:09.329902 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-grvw8"] Jul 22 02:20:09.333254 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:20:09.333223 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25bca1ce_620d_4f9b_8f60_279b6e21668e.slice/crio-b6aabf8828b43ff90a24e76a82ff9463cbf6ba0afed87944d3e097617eefb980 WatchSource:0}: Error finding container b6aabf8828b43ff90a24e76a82ff9463cbf6ba0afed87944d3e097617eefb980: Status 404 returned error can't find the container with id b6aabf8828b43ff90a24e76a82ff9463cbf6ba0afed87944d3e097617eefb980 Jul 22 02:20:09.402431 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:09.402404 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rzszs_519f47d6-aef0-4eae-8eba-240fe5cadfed/node-ca/0.log" Jul 22 02:20:09.720388 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:09.720350 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:09.721000 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:09.720394 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls\") pod \"dns-default-zgnxf\" (UID: \"c3fb8521-f63a-4773-ad73-73ae63e8b4d2\") " pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:09.721000 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:09.720498 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:20:09.721000 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:09.720513 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c6f8455b-2hn2c: secret "image-registry-tls" not found Jul 22 02:20:09.721000 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:09.720541 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:20:09.721000 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:09.720575 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls podName:b6e35b92-2a47-4594-8b5e-9e8110892637 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:13.720558173 +0000 UTC m=+44.629423161 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls") pod "image-registry-84c6f8455b-2hn2c" (UID: "b6e35b92-2a47-4594-8b5e-9e8110892637") : secret "image-registry-tls" not found Jul 22 02:20:09.721000 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:09.720596 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls podName:c3fb8521-f63a-4773-ad73-73ae63e8b4d2 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:13.720581775 +0000 UTC m=+44.629446748 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls") pod "dns-default-zgnxf" (UID: "c3fb8521-f63a-4773-ad73-73ae63e8b4d2") : secret "dns-default-metrics-tls" not found Jul 22 02:20:09.876893 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:09.876793 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mwwl8" event={"ID":"1a22a50d-1436-409d-af35-9dc1907dd74e","Type":"ContainerStarted","Data":"1583f28140cf1676f89feaf899b91b72d803e8211453f2c3ba7c837dd1ae7e25"} Jul 22 02:20:09.880016 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:09.879960 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m5st4" event={"ID":"a16f14c0-d854-40e4-8151-abebbe9b1176","Type":"ContainerStarted","Data":"7015f8b3ee81b864e34c49b54d48284ac4d6a0bca8180cc1fc2758284136b2b1"} Jul 22 02:20:09.881052 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:09.881029 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-grvw8" event={"ID":"25bca1ce-620d-4f9b-8f60-279b6e21668e","Type":"ContainerStarted","Data":"b6aabf8828b43ff90a24e76a82ff9463cbf6ba0afed87944d3e097617eefb980"} Jul 22 02:20:09.893454 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:09.893407 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mwwl8" podStartSLOduration=2.20886578 podStartE2EDuration="4.893393804s" totalImagesPullingTime="2.684528024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:09.893119548 +0000 UTC m=+40.801984545" watchObservedRunningTime="2026-07-22 02:20:09.893393804 +0000 UTC m=+40.802258798" Jul 22 02:20:09.917734 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:09.917694 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-m5st4" podStartSLOduration=8.75209982 podStartE2EDuration="39.917677458s" totalImagesPullingTime="31.165577638s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:09.917122285 +0000 UTC m=+40.825987281" watchObservedRunningTime="2026-07-22 02:20:09.917677458 +0000 UTC m=+40.826542471" Jul 22 02:20:10.167182 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.167096 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-rxrk5"] Jul 22 02:20:10.191307 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.191281 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-rxrk5"] Jul 22 02:20:10.191458 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.191375 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-rxrk5" Jul 22 02:20:10.194193 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.194165 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 22 02:20:10.194193 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.194182 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-t5p4w\"" Jul 22 02:20:10.194402 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.194269 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 22 02:20:10.194402 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.194176 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 22 02:20:10.194493 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.194421 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 22 02:20:10.202067 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.202049 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mwwl8_1a22a50d-1436-409d-af35-9dc1907dd74e/serve-healthcheck-canary/0.log" Jul 22 02:20:10.325231 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.325198 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rfst\" (UniqueName: \"kubernetes.io/projected/6cfdc000-2a55-4893-ae66-62d4289af261-kube-api-access-7rfst\") pod \"service-ca-58598695f7-rxrk5\" (UID: \"6cfdc000-2a55-4893-ae66-62d4289af261\") " pod="openshift-service-ca/service-ca-58598695f7-rxrk5" Jul 22 02:20:10.325383 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.325260 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6cfdc000-2a55-4893-ae66-62d4289af261-signing-cabundle\") pod \"service-ca-58598695f7-rxrk5\" (UID: \"6cfdc000-2a55-4893-ae66-62d4289af261\") " pod="openshift-service-ca/service-ca-58598695f7-rxrk5" Jul 22 02:20:10.325383 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.325365 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6cfdc000-2a55-4893-ae66-62d4289af261-signing-key\") pod \"service-ca-58598695f7-rxrk5\" (UID: \"6cfdc000-2a55-4893-ae66-62d4289af261\") " pod="openshift-service-ca/service-ca-58598695f7-rxrk5" Jul 22 02:20:10.426632 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.426548 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6cfdc000-2a55-4893-ae66-62d4289af261-signing-key\") pod \"service-ca-58598695f7-rxrk5\" (UID: \"6cfdc000-2a55-4893-ae66-62d4289af261\") " pod="openshift-service-ca/service-ca-58598695f7-rxrk5" Jul 22 02:20:10.426632 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.426617 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rfst\" (UniqueName: \"kubernetes.io/projected/6cfdc000-2a55-4893-ae66-62d4289af261-kube-api-access-7rfst\") pod \"service-ca-58598695f7-rxrk5\" (UID: \"6cfdc000-2a55-4893-ae66-62d4289af261\") " pod="openshift-service-ca/service-ca-58598695f7-rxrk5" Jul 22 02:20:10.426955 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.426681 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6cfdc000-2a55-4893-ae66-62d4289af261-signing-cabundle\") pod \"service-ca-58598695f7-rxrk5\" (UID: \"6cfdc000-2a55-4893-ae66-62d4289af261\") " pod="openshift-service-ca/service-ca-58598695f7-rxrk5" Jul 22 02:20:10.427589 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.427558 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6cfdc000-2a55-4893-ae66-62d4289af261-signing-cabundle\") pod \"service-ca-58598695f7-rxrk5\" (UID: \"6cfdc000-2a55-4893-ae66-62d4289af261\") " pod="openshift-service-ca/service-ca-58598695f7-rxrk5" Jul 22 02:20:10.430564 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.430543 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6cfdc000-2a55-4893-ae66-62d4289af261-signing-key\") pod \"service-ca-58598695f7-rxrk5\" (UID: \"6cfdc000-2a55-4893-ae66-62d4289af261\") " pod="openshift-service-ca/service-ca-58598695f7-rxrk5" Jul 22 02:20:10.436600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.436577 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rfst\" (UniqueName: \"kubernetes.io/projected/6cfdc000-2a55-4893-ae66-62d4289af261-kube-api-access-7rfst\") pod \"service-ca-58598695f7-rxrk5\" (UID: \"6cfdc000-2a55-4893-ae66-62d4289af261\") " pod="openshift-service-ca/service-ca-58598695f7-rxrk5" Jul 22 02:20:10.502934 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.502862 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-rxrk5" Jul 22 02:20:10.629220 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.629188 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/13baf0c2-a61b-4602-970d-6bde2895bc5e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:10.629366 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.629224 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:10.629366 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:10.629337 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:20:10.629454 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:10.629391 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls podName:13baf0c2-a61b-4602-970d-6bde2895bc5e nodeName:}" failed. No retries permitted until 2026-07-22 02:20:14.629378393 +0000 UTC m=+45.538243366 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gmw9c" (UID: "13baf0c2-a61b-4602-970d-6bde2895bc5e") : secret "insights-runtime-extractor-tls" not found Jul 22 02:20:10.629740 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.629716 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/13baf0c2-a61b-4602-970d-6bde2895bc5e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:10.712582 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.712553 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-rxrk5"] Jul 22 02:20:10.716091 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:20:10.716065 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cfdc000_2a55_4893_ae66_62d4289af261.slice/crio-f33787cf7baaeab7c2c8e785395d56c1df78e3f2b3ea1b592d815630af56cdc8 WatchSource:0}: Error finding container f33787cf7baaeab7c2c8e785395d56c1df78e3f2b3ea1b592d815630af56cdc8: Status 404 returned error can't find the container with id f33787cf7baaeab7c2c8e785395d56c1df78e3f2b3ea1b592d815630af56cdc8 Jul 22 02:20:10.884379 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:10.884347 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-rxrk5" event={"ID":"6cfdc000-2a55-4893-ae66-62d4289af261","Type":"ContainerStarted","Data":"f33787cf7baaeab7c2c8e785395d56c1df78e3f2b3ea1b592d815630af56cdc8"} Jul 22 02:20:11.889214 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:11.889173 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-grvw8" event={"ID":"25bca1ce-620d-4f9b-8f60-279b6e21668e","Type":"ContainerStarted","Data":"f34d2379407e149e5a6f01d557a6febe5ec8134819f1d0359dae1cba1a72e11f"} Jul 22 02:20:11.889675 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:11.889238 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-grvw8" event={"ID":"25bca1ce-620d-4f9b-8f60-279b6e21668e","Type":"ContainerStarted","Data":"39186cf735a7a53f4fb4d7c14922157cb0b2a46f377e27f9659504dfcee720f6"} Jul 22 02:20:11.908966 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:11.908910 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-grvw8" podStartSLOduration=2.308128994 podStartE2EDuration="3.908895026s" totalImagesPullingTime="1.600766032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:11.907810985 +0000 UTC m=+42.816675980" watchObservedRunningTime="2026-07-22 02:20:11.908895026 +0000 UTC m=+42.817760043" Jul 22 02:20:13.753220 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:13.753185 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:13.753567 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:13.753238 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls\") pod \"dns-default-zgnxf\" (UID: \"c3fb8521-f63a-4773-ad73-73ae63e8b4d2\") " pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:13.753567 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:13.753344 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:20:13.753567 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:13.753358 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c6f8455b-2hn2c: secret "image-registry-tls" not found Jul 22 02:20:13.753567 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:13.753410 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls podName:b6e35b92-2a47-4594-8b5e-9e8110892637 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:21.753397227 +0000 UTC m=+52.662262199 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls") pod "image-registry-84c6f8455b-2hn2c" (UID: "b6e35b92-2a47-4594-8b5e-9e8110892637") : secret "image-registry-tls" not found Jul 22 02:20:13.753567 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:13.753421 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:20:13.753567 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:13.753479 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls podName:c3fb8521-f63a-4773-ad73-73ae63e8b4d2 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:21.753460144 +0000 UTC m=+52.662325122 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls") pod "dns-default-zgnxf" (UID: "c3fb8521-f63a-4773-ad73-73ae63e8b4d2") : secret "dns-default-metrics-tls" not found Jul 22 02:20:13.895341 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:13.895303 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-rxrk5" event={"ID":"6cfdc000-2a55-4893-ae66-62d4289af261","Type":"ContainerStarted","Data":"61b5bbfe8ab289cf47ed25585a48a077d9bfb9723ea5f1b213919dc3fbeeac2c"} Jul 22 02:20:13.915204 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:13.915152 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-rxrk5" podStartSLOduration=1.405307991 podStartE2EDuration="3.915134945s" totalImagesPullingTime="2.509826954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:13.914514492 +0000 UTC m=+44.823379487" watchObservedRunningTime="2026-07-22 02:20:13.915134945 +0000 UTC m=+44.823999940" Jul 22 02:20:14.661433 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:14.661383 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:14.661605 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:14.661559 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:20:14.661685 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:20:14.661636 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls podName:13baf0c2-a61b-4602-970d-6bde2895bc5e nodeName:}" failed. No retries permitted until 2026-07-22 02:20:22.661616089 +0000 UTC m=+53.570481062 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gmw9c" (UID: "13baf0c2-a61b-4602-970d-6bde2895bc5e") : secret "insights-runtime-extractor-tls" not found Jul 22 02:20:21.813745 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:21.813697 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:21.813745 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:21.813747 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls\") pod \"dns-default-zgnxf\" (UID: \"c3fb8521-f63a-4773-ad73-73ae63e8b4d2\") " pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:21.816204 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:21.816170 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3fb8521-f63a-4773-ad73-73ae63e8b4d2-metrics-tls\") pod \"dns-default-zgnxf\" (UID: \"c3fb8521-f63a-4773-ad73-73ae63e8b4d2\") " pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:21.816365 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:21.816343 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls\") pod \"image-registry-84c6f8455b-2hn2c\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:21.863323 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:21.863295 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:21.876449 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:21.876421 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:22.020343 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:22.020314 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84c6f8455b-2hn2c"] Jul 22 02:20:22.023920 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:20:22.023894 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e35b92_2a47_4594_8b5e_9e8110892637.slice/crio-fb9eaa74156babca9f747ea626735a494d4bd2573d6f55773b48cb11b06d0147 WatchSource:0}: Error finding container fb9eaa74156babca9f747ea626735a494d4bd2573d6f55773b48cb11b06d0147: Status 404 returned error can't find the container with id fb9eaa74156babca9f747ea626735a494d4bd2573d6f55773b48cb11b06d0147 Jul 22 02:20:22.050351 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:22.047940 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zgnxf"] Jul 22 02:20:22.052698 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:20:22.052666 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3fb8521_f63a_4773_ad73_73ae63e8b4d2.slice/crio-7ca79da60a9dafc12dc3add025a84f47842e81cf36f1261bb724a5e6bb76ce5e WatchSource:0}: Error finding container 7ca79da60a9dafc12dc3add025a84f47842e81cf36f1261bb724a5e6bb76ce5e: Status 404 returned error can't find the container with id 7ca79da60a9dafc12dc3add025a84f47842e81cf36f1261bb724a5e6bb76ce5e Jul 22 02:20:22.720173 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:22.720132 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:22.722569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:22.722541 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/13baf0c2-a61b-4602-970d-6bde2895bc5e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gmw9c\" (UID: \"13baf0c2-a61b-4602-970d-6bde2895bc5e\") " pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:22.736498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:22.736475 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gmw9c" Jul 22 02:20:22.899721 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:22.899400 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gmw9c"] Jul 22 02:20:22.902044 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:20:22.902023 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13baf0c2_a61b_4602_970d_6bde2895bc5e.slice/crio-02dfa213d1d26fe78a6afd6b62a5e35768c263eb87170025df4901e478805f46 WatchSource:0}: Error finding container 02dfa213d1d26fe78a6afd6b62a5e35768c263eb87170025df4901e478805f46: Status 404 returned error can't find the container with id 02dfa213d1d26fe78a6afd6b62a5e35768c263eb87170025df4901e478805f46 Jul 22 02:20:22.919627 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:22.919592 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gmw9c" event={"ID":"13baf0c2-a61b-4602-970d-6bde2895bc5e","Type":"ContainerStarted","Data":"02dfa213d1d26fe78a6afd6b62a5e35768c263eb87170025df4901e478805f46"} Jul 22 02:20:22.921203 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:22.921177 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" event={"ID":"b6e35b92-2a47-4594-8b5e-9e8110892637","Type":"ContainerStarted","Data":"1e5aff9999412c965e9a54d2fe13123a9f630480e227e9ee8f8d48646b0577c1"} Jul 22 02:20:22.921334 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:22.921221 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" event={"ID":"b6e35b92-2a47-4594-8b5e-9e8110892637","Type":"ContainerStarted","Data":"fb9eaa74156babca9f747ea626735a494d4bd2573d6f55773b48cb11b06d0147"} Jul 22 02:20:22.921334 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:22.921271 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:22.922302 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:22.922280 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zgnxf" event={"ID":"c3fb8521-f63a-4773-ad73-73ae63e8b4d2","Type":"ContainerStarted","Data":"7ca79da60a9dafc12dc3add025a84f47842e81cf36f1261bb724a5e6bb76ce5e"} Jul 22 02:20:22.943582 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:22.943536 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" podStartSLOduration=24.943523671 podStartE2EDuration="24.943523671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:22.943076515 +0000 UTC m=+53.851941512" watchObservedRunningTime="2026-07-22 02:20:22.943523671 +0000 UTC m=+53.852388666" Jul 22 02:20:24.932121 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:24.932077 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zgnxf" event={"ID":"c3fb8521-f63a-4773-ad73-73ae63e8b4d2","Type":"ContainerStarted","Data":"d5d104334edc0e061a935e95d1db032c29fb011790999f0688de705b135d25f4"} Jul 22 02:20:24.932599 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:24.932142 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zgnxf" event={"ID":"c3fb8521-f63a-4773-ad73-73ae63e8b4d2","Type":"ContainerStarted","Data":"525871f60ef76057711017e39c62d912f35e377b44fb2b19195d6f6b684eb671"} Jul 22 02:20:24.932599 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:24.932361 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:24.953405 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:24.953352 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zgnxf" podStartSLOduration=17.652545787 podStartE2EDuration="19.953332991s" totalImagesPullingTime="2.300787204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:24.952841398 +0000 UTC m=+55.861706390" watchObservedRunningTime="2026-07-22 02:20:24.953332991 +0000 UTC m=+55.862197987" Jul 22 02:20:28.950515 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:28.950433 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gmw9c" event={"ID":"13baf0c2-a61b-4602-970d-6bde2895bc5e","Type":"ContainerStarted","Data":"1656fc15fda2aa16b927b87d0cfabdd312679ca977ab843472b8f013a86aeec6"} Jul 22 02:20:30.853008 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:30.852972 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zf29q" Jul 22 02:20:31.962092 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:31.962057 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gmw9c" event={"ID":"13baf0c2-a61b-4602-970d-6bde2895bc5e","Type":"ContainerStarted","Data":"608ecddcce53968080b96f24af32ba2fab3f79077c8bb59877f2f47abfdd77ee"} Jul 22 02:20:34.939380 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:34.939348 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zgnxf" Jul 22 02:20:35.299794 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.299761 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-xsc6x"] Jul 22 02:20:35.319847 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.319766 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt"] Jul 22 02:20:35.320010 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.319934 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xsc6x" Jul 22 02:20:35.322855 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.322801 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret\") pod \"global-pull-secret-syncer-w4bgx\" (UID: \"ede3658a-126b-446f-b57a-678aefbc7936\") " pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:20:35.322988 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.322925 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 22 02:20:35.323108 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.323088 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xxnhv\"" Jul 22 02:20:35.323341 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.323310 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 22 02:20:35.323453 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.323358 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84c6f8455b-2hn2c"] Jul 22 02:20:35.323453 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.323400 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn"] Jul 22 02:20:35.323570 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.323553 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt" Jul 22 02:20:35.326422 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.325768 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt"] Jul 22 02:20:35.326422 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.325805 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-xsc6x"] Jul 22 02:20:35.326422 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.325918 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.327641 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.327464 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 22 02:20:35.327641 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.327579 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 02:20:35.327810 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.327713 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 22 02:20:35.329464 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.329439 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 22 02:20:35.329563 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.329527 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 22 02:20:35.329678 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.329661 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 22 02:20:35.329745 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.329664 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 22 02:20:35.330685 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.330666 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 22 02:20:35.330685 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.330682 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 22 02:20:35.336768 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.336712 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ede3658a-126b-446f-b57a-678aefbc7936-original-pull-secret\") pod \"global-pull-secret-syncer-w4bgx\" (UID: \"ede3658a-126b-446f-b57a-678aefbc7936\") " pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:20:35.339463 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.339409 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn"] Jul 22 02:20:35.409316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.409286 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-68dcf7c957-6cgls"] Jul 22 02:20:35.411344 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.411326 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.413335 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.413318 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w4bgx" Jul 22 02:20:35.423596 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.423572 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5781ae83-c632-4fcf-8705-3e9dc64dab24-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.423690 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.423614 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs\") pod \"network-metrics-daemon-mxnss\" (UID: \"72a0ac8c-4aaf-45c2-bd02-41b8e80e770b\") " pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:20:35.423690 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.423631 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/446feb28-20e9-4ab6-a1c4-f03df2cde5e4-tmp\") pod \"klusterlet-addon-workmgr-6db6f98976-b9ppt\" (UID: \"446feb28-20e9-4ab6-a1c4-f03df2cde5e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt" Jul 22 02:20:35.423690 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.423658 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/28e31c18-494c-4fd8-896d-b302c92de0a6-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xsc6x\" (UID: \"28e31c18-494c-4fd8-896d-b302c92de0a6\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xsc6x" Jul 22 02:20:35.423690 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.423685 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr9mr\" (UniqueName: \"kubernetes.io/projected/446feb28-20e9-4ab6-a1c4-f03df2cde5e4-kube-api-access-fr9mr\") pod \"klusterlet-addon-workmgr-6db6f98976-b9ppt\" (UID: \"446feb28-20e9-4ab6-a1c4-f03df2cde5e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt" Jul 22 02:20:35.423902 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.423708 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5781ae83-c632-4fcf-8705-3e9dc64dab24-ca\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.423902 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.423734 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/28e31c18-494c-4fd8-896d-b302c92de0a6-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-xsc6x\" (UID: \"28e31c18-494c-4fd8-896d-b302c92de0a6\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xsc6x" Jul 22 02:20:35.423902 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.423757 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5781ae83-c632-4fcf-8705-3e9dc64dab24-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.423902 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.423818 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5781ae83-c632-4fcf-8705-3e9dc64dab24-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.423902 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.423884 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/446feb28-20e9-4ab6-a1c4-f03df2cde5e4-klusterlet-config\") pod \"klusterlet-addon-workmgr-6db6f98976-b9ppt\" (UID: \"446feb28-20e9-4ab6-a1c4-f03df2cde5e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt" Jul 22 02:20:35.424118 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.423906 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5781ae83-c632-4fcf-8705-3e9dc64dab24-hub\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.424118 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.423941 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgbcv\" (UniqueName: \"kubernetes.io/projected/5781ae83-c632-4fcf-8705-3e9dc64dab24-kube-api-access-jgbcv\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.424118 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.423962 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6gfw\" (UniqueName: \"kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw\") pod \"network-check-target-tqrxj\" (UID: \"d8fbc209-ac7d-480a-b8e2-3e4083f94ec9\") " pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:20:35.426256 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.426238 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 02:20:35.427216 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.427199 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 02:20:35.436008 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.435989 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72a0ac8c-4aaf-45c2-bd02-41b8e80e770b-metrics-certs\") pod \"network-metrics-daemon-mxnss\" (UID: \"72a0ac8c-4aaf-45c2-bd02-41b8e80e770b\") " pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:20:35.436520 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.436502 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 02:20:35.441666 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.441647 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68dcf7c957-6cgls"] Jul 22 02:20:35.448723 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.448703 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6gfw\" (UniqueName: \"kubernetes.io/projected/d8fbc209-ac7d-480a-b8e2-3e4083f94ec9-kube-api-access-k6gfw\") pod \"network-check-target-tqrxj\" (UID: \"d8fbc209-ac7d-480a-b8e2-3e4083f94ec9\") " pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:20:35.522589 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.522564 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t8kv7\"" Jul 22 02:20:35.525014 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.524991 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/28e31c18-494c-4fd8-896d-b302c92de0a6-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-xsc6x\" (UID: \"28e31c18-494c-4fd8-896d-b302c92de0a6\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xsc6x" Jul 22 02:20:35.525133 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525026 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5781ae83-c632-4fcf-8705-3e9dc64dab24-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.525208 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525151 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d797fd6-18bb-420d-b40a-89b76860ae09-trusted-ca\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.525264 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525203 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtkwf\" (UniqueName: \"kubernetes.io/projected/7d797fd6-18bb-420d-b40a-89b76860ae09-kube-api-access-dtkwf\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.525264 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525239 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d797fd6-18bb-420d-b40a-89b76860ae09-installation-pull-secrets\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.525368 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525276 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5781ae83-c632-4fcf-8705-3e9dc64dab24-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.525368 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525312 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d797fd6-18bb-420d-b40a-89b76860ae09-image-registry-private-configuration\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.525368 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525340 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d797fd6-18bb-420d-b40a-89b76860ae09-ca-trust-extracted\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.525501 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525368 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d797fd6-18bb-420d-b40a-89b76860ae09-bound-sa-token\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.525501 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525440 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/446feb28-20e9-4ab6-a1c4-f03df2cde5e4-klusterlet-config\") pod \"klusterlet-addon-workmgr-6db6f98976-b9ppt\" (UID: \"446feb28-20e9-4ab6-a1c4-f03df2cde5e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt" Jul 22 02:20:35.525501 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525464 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5781ae83-c632-4fcf-8705-3e9dc64dab24-hub\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.525501 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525486 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgbcv\" (UniqueName: \"kubernetes.io/projected/5781ae83-c632-4fcf-8705-3e9dc64dab24-kube-api-access-jgbcv\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.525741 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525718 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d797fd6-18bb-420d-b40a-89b76860ae09-registry-certificates\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.525844 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525739 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/28e31c18-494c-4fd8-896d-b302c92de0a6-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-xsc6x\" (UID: \"28e31c18-494c-4fd8-896d-b302c92de0a6\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xsc6x" Jul 22 02:20:35.525844 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525806 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5781ae83-c632-4fcf-8705-3e9dc64dab24-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.525919 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525877 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d797fd6-18bb-420d-b40a-89b76860ae09-registry-tls\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.525919 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525909 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/446feb28-20e9-4ab6-a1c4-f03df2cde5e4-tmp\") pod \"klusterlet-addon-workmgr-6db6f98976-b9ppt\" (UID: \"446feb28-20e9-4ab6-a1c4-f03df2cde5e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt" Jul 22 02:20:35.525978 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525936 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/28e31c18-494c-4fd8-896d-b302c92de0a6-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xsc6x\" (UID: \"28e31c18-494c-4fd8-896d-b302c92de0a6\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xsc6x" Jul 22 02:20:35.525978 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525965 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr9mr\" (UniqueName: \"kubernetes.io/projected/446feb28-20e9-4ab6-a1c4-f03df2cde5e4-kube-api-access-fr9mr\") pod \"klusterlet-addon-workmgr-6db6f98976-b9ppt\" (UID: \"446feb28-20e9-4ab6-a1c4-f03df2cde5e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt" Jul 22 02:20:35.526040 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.525989 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5781ae83-c632-4fcf-8705-3e9dc64dab24-ca\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.526286 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.526263 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5781ae83-c632-4fcf-8705-3e9dc64dab24-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.526673 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.526593 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/446feb28-20e9-4ab6-a1c4-f03df2cde5e4-tmp\") pod \"klusterlet-addon-workmgr-6db6f98976-b9ppt\" (UID: \"446feb28-20e9-4ab6-a1c4-f03df2cde5e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt" Jul 22 02:20:35.527131 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.527110 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8wzn5\"" Jul 22 02:20:35.528288 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.528258 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5781ae83-c632-4fcf-8705-3e9dc64dab24-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.528525 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.528503 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5781ae83-c632-4fcf-8705-3e9dc64dab24-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.528814 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.528794 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/446feb28-20e9-4ab6-a1c4-f03df2cde5e4-klusterlet-config\") pod \"klusterlet-addon-workmgr-6db6f98976-b9ppt\" (UID: \"446feb28-20e9-4ab6-a1c4-f03df2cde5e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt" Jul 22 02:20:35.528814 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.528806 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5781ae83-c632-4fcf-8705-3e9dc64dab24-hub\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.529130 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.529108 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/28e31c18-494c-4fd8-896d-b302c92de0a6-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xsc6x\" (UID: \"28e31c18-494c-4fd8-896d-b302c92de0a6\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xsc6x" Jul 22 02:20:35.530081 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.530064 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5781ae83-c632-4fcf-8705-3e9dc64dab24-ca\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.530893 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.530874 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mxnss" Jul 22 02:20:35.535616 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.535594 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:20:35.536421 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.536396 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr9mr\" (UniqueName: \"kubernetes.io/projected/446feb28-20e9-4ab6-a1c4-f03df2cde5e4-kube-api-access-fr9mr\") pod \"klusterlet-addon-workmgr-6db6f98976-b9ppt\" (UID: \"446feb28-20e9-4ab6-a1c4-f03df2cde5e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt" Jul 22 02:20:35.536666 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.536649 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgbcv\" (UniqueName: \"kubernetes.io/projected/5781ae83-c632-4fcf-8705-3e9dc64dab24-kube-api-access-jgbcv\") pod \"cluster-proxy-proxy-agent-6b5dfbd58f-45jnn\" (UID: \"5781ae83-c632-4fcf-8705-3e9dc64dab24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.560895 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.560864 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w4bgx"] Jul 22 02:20:35.563017 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:20:35.562992 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podede3658a_126b_446f_b57a_678aefbc7936.slice/crio-cc786b7ac9d4bc8994a67b42d9d3f6902c5324efe6ce0d0806f8048ea660b256 WatchSource:0}: Error finding container cc786b7ac9d4bc8994a67b42d9d3f6902c5324efe6ce0d0806f8048ea660b256: Status 404 returned error can't find the container with id cc786b7ac9d4bc8994a67b42d9d3f6902c5324efe6ce0d0806f8048ea660b256 Jul 22 02:20:35.627266 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.627240 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d797fd6-18bb-420d-b40a-89b76860ae09-trusted-ca\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.627372 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.627273 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtkwf\" (UniqueName: \"kubernetes.io/projected/7d797fd6-18bb-420d-b40a-89b76860ae09-kube-api-access-dtkwf\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.627372 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.627298 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d797fd6-18bb-420d-b40a-89b76860ae09-installation-pull-secrets\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.627372 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.627328 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d797fd6-18bb-420d-b40a-89b76860ae09-image-registry-private-configuration\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.627372 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.627352 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d797fd6-18bb-420d-b40a-89b76860ae09-ca-trust-extracted\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.627580 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.627376 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d797fd6-18bb-420d-b40a-89b76860ae09-bound-sa-token\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.627580 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.627432 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d797fd6-18bb-420d-b40a-89b76860ae09-registry-certificates\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.627580 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.627486 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d797fd6-18bb-420d-b40a-89b76860ae09-registry-tls\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.628413 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.628364 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d797fd6-18bb-420d-b40a-89b76860ae09-registry-certificates\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.628413 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.628395 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d797fd6-18bb-420d-b40a-89b76860ae09-ca-trust-extracted\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.628747 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.628473 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d797fd6-18bb-420d-b40a-89b76860ae09-trusted-ca\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.630418 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.630391 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d797fd6-18bb-420d-b40a-89b76860ae09-installation-pull-secrets\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.630591 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.630574 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d797fd6-18bb-420d-b40a-89b76860ae09-image-registry-private-configuration\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.631055 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.631039 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d797fd6-18bb-420d-b40a-89b76860ae09-registry-tls\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.632805 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.632791 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xsc6x" Jul 22 02:20:35.635924 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.635903 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtkwf\" (UniqueName: \"kubernetes.io/projected/7d797fd6-18bb-420d-b40a-89b76860ae09-kube-api-access-dtkwf\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.639670 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.636906 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d797fd6-18bb-420d-b40a-89b76860ae09-bound-sa-token\") pod \"image-registry-68dcf7c957-6cgls\" (UID: \"7d797fd6-18bb-420d-b40a-89b76860ae09\") " pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.649184 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.649161 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt" Jul 22 02:20:35.670511 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.670486 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" Jul 22 02:20:35.674696 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.674615 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mxnss"] Jul 22 02:20:35.679773 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:20:35.679734 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72a0ac8c_4aaf_45c2_bd02_41b8e80e770b.slice/crio-b44ee32ac77f916ab66b5361ff4913fd49aa91808edeefda7877ca8ae8acad17 WatchSource:0}: Error finding container b44ee32ac77f916ab66b5361ff4913fd49aa91808edeefda7877ca8ae8acad17: Status 404 returned error can't find the container with id b44ee32ac77f916ab66b5361ff4913fd49aa91808edeefda7877ca8ae8acad17 Jul 22 02:20:35.697430 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.697356 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tqrxj"] Jul 22 02:20:35.720401 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.720313 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.793620 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.793588 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-xsc6x"] Jul 22 02:20:35.798476 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:20:35.798441 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28e31c18_494c_4fd8_896d_b302c92de0a6.slice/crio-f6d73ff43597e5be8000685d9be401039bf1c83cdae6228a0797ec7411f1c1b1 WatchSource:0}: Error finding container f6d73ff43597e5be8000685d9be401039bf1c83cdae6228a0797ec7411f1c1b1: Status 404 returned error can't find the container with id f6d73ff43597e5be8000685d9be401039bf1c83cdae6228a0797ec7411f1c1b1 Jul 22 02:20:35.816763 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.816673 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt"] Jul 22 02:20:35.821890 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:20:35.821857 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod446feb28_20e9_4ab6_a1c4_f03df2cde5e4.slice/crio-3a08ea1080bbdcda1e4b9925b8a85285573d2ab0db4583cddc86408ca2091afc WatchSource:0}: Error finding container 3a08ea1080bbdcda1e4b9925b8a85285573d2ab0db4583cddc86408ca2091afc: Status 404 returned error can't find the container with id 3a08ea1080bbdcda1e4b9925b8a85285573d2ab0db4583cddc86408ca2091afc Jul 22 02:20:35.851056 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.851029 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn"] Jul 22 02:20:35.881667 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.881638 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68dcf7c957-6cgls"] Jul 22 02:20:35.884371 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:20:35.884349 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d797fd6_18bb_420d_b40a_89b76860ae09.slice/crio-005d71eb8c9df5cc7b6ca211c92f4de34b451a4609963adc288d3df574f0d0da WatchSource:0}: Error finding container 005d71eb8c9df5cc7b6ca211c92f4de34b451a4609963adc288d3df574f0d0da: Status 404 returned error can't find the container with id 005d71eb8c9df5cc7b6ca211c92f4de34b451a4609963adc288d3df574f0d0da Jul 22 02:20:35.976394 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.976361 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w4bgx" event={"ID":"ede3658a-126b-446f-b57a-678aefbc7936","Type":"ContainerStarted","Data":"cc786b7ac9d4bc8994a67b42d9d3f6902c5324efe6ce0d0806f8048ea660b256"} Jul 22 02:20:35.978523 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.978496 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gmw9c" event={"ID":"13baf0c2-a61b-4602-970d-6bde2895bc5e","Type":"ContainerStarted","Data":"61464a720e3e14b2db5bd1ecc678b450cebb60269aa057a06261e23938861573"} Jul 22 02:20:35.980026 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.980003 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" event={"ID":"7d797fd6-18bb-420d-b40a-89b76860ae09","Type":"ContainerStarted","Data":"a35c93c731653ba097f1e6051df4075f181cc817328cdb383e8788d7b4583f2a"} Jul 22 02:20:35.980119 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.980040 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" event={"ID":"7d797fd6-18bb-420d-b40a-89b76860ae09","Type":"ContainerStarted","Data":"005d71eb8c9df5cc7b6ca211c92f4de34b451a4609963adc288d3df574f0d0da"} Jul 22 02:20:35.980119 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.980077 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:35.981137 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.981114 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" event={"ID":"5781ae83-c632-4fcf-8705-3e9dc64dab24","Type":"ContainerStarted","Data":"c9c9053e108a16d2a08d70970a781b5b3c1ae61af2e2eba1616275ef0513bc72"} Jul 22 02:20:35.982409 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.982384 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt" event={"ID":"446feb28-20e9-4ab6-a1c4-f03df2cde5e4","Type":"ContainerStarted","Data":"3a08ea1080bbdcda1e4b9925b8a85285573d2ab0db4583cddc86408ca2091afc"} Jul 22 02:20:35.983614 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.983590 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xsc6x" event={"ID":"28e31c18-494c-4fd8-896d-b302c92de0a6","Type":"ContainerStarted","Data":"f6d73ff43597e5be8000685d9be401039bf1c83cdae6228a0797ec7411f1c1b1"} Jul 22 02:20:35.984789 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.984767 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mxnss" event={"ID":"72a0ac8c-4aaf-45c2-bd02-41b8e80e770b","Type":"ContainerStarted","Data":"b44ee32ac77f916ab66b5361ff4913fd49aa91808edeefda7877ca8ae8acad17"} Jul 22 02:20:35.985863 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.985789 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tqrxj" event={"ID":"d8fbc209-ac7d-480a-b8e2-3e4083f94ec9","Type":"ContainerStarted","Data":"93859e600372caabf060981b397b084c7a833f56443068f1de3cc0d86cef60d3"} Jul 22 02:20:35.999589 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:35.999540 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gmw9c" podStartSLOduration=17.572228276 podStartE2EDuration="29.999524761s" totalImagesPullingTime="12.427296485s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:20:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:35.998879577 +0000 UTC m=+66.907744575" watchObservedRunningTime="2026-07-22 02:20:35.999524761 +0000 UTC m=+66.908389759" Jul 22 02:20:36.018975 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:36.018935 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" podStartSLOduration=1.018919262 podStartE2EDuration="1.018919262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:36.018159338 +0000 UTC m=+66.927024334" watchObservedRunningTime="2026-07-22 02:20:36.018919262 +0000 UTC m=+66.927784255" Jul 22 02:20:39.012152 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:39.012112 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xsc6x" event={"ID":"28e31c18-494c-4fd8-896d-b302c92de0a6","Type":"ContainerStarted","Data":"8e2f976d58469fcfa2d3ef2c9e471234d325693eab25dfa2d69f9e4f5b510d7a"} Jul 22 02:20:39.017056 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:39.017021 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mxnss" event={"ID":"72a0ac8c-4aaf-45c2-bd02-41b8e80e770b","Type":"ContainerStarted","Data":"ffb959b2597144cde22e7935dca0b8acc6bf3121bae3a698167cfed21a9a918a"} Jul 22 02:20:39.017219 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:39.017086 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mxnss" event={"ID":"72a0ac8c-4aaf-45c2-bd02-41b8e80e770b","Type":"ContainerStarted","Data":"7c962f77e3650c8c86d0d5597d53e249859b3b96edda4bfd3ac5599901da6416"} Jul 22 02:20:39.037217 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:39.037149 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xsc6x" podStartSLOduration=1.890809207 podStartE2EDuration="4.037130584s" totalImagesPullingTime="2.146321377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:39.035766487 +0000 UTC m=+69.944631483" watchObservedRunningTime="2026-07-22 02:20:39.037130584 +0000 UTC m=+69.945995583" Jul 22 02:20:44.040057 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:44.039959 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tqrxj" event={"ID":"d8fbc209-ac7d-480a-b8e2-3e4083f94ec9","Type":"ContainerStarted","Data":"6df7e7fdf97a8e20a4f3b704c418fcf0a17c49de28f9c147f76886235057e45e"} Jul 22 02:20:44.040057 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:44.040058 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:20:44.041311 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:44.041288 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w4bgx" event={"ID":"ede3658a-126b-446f-b57a-678aefbc7936","Type":"ContainerStarted","Data":"ec8af06077993d0a6bd8ec533d78fedcbbde23f54af0ea228fef4d8b05e18a00"} Jul 22 02:20:44.055734 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:44.055688 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tqrxj" podStartSLOduration=66.032580262 podStartE2EDuration="1m14.055669582s" totalImagesPullingTime="8.02308932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:44.055410161 +0000 UTC m=+74.964275156" watchObservedRunningTime="2026-07-22 02:20:44.055669582 +0000 UTC m=+74.964534577" Jul 22 02:20:44.055882 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:44.055866 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mxnss" podStartSLOduration=71.790290692 podStartE2EDuration="1m14.05586032s" totalImagesPullingTime="2.265569628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:39.061990382 +0000 UTC m=+69.970855377" watchObservedRunningTime="2026-07-22 02:20:44.05586032 +0000 UTC m=+74.964725340" Jul 22 02:20:44.069672 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:44.069624 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-w4bgx" podStartSLOduration=65.891749957 podStartE2EDuration="1m14.069606527s" totalImagesPullingTime="8.17785657s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:44.069306837 +0000 UTC m=+74.978171833" watchObservedRunningTime="2026-07-22 02:20:44.069606527 +0000 UTC m=+74.978471526" Jul 22 02:20:45.047269 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:45.047234 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" event={"ID":"5781ae83-c632-4fcf-8705-3e9dc64dab24","Type":"ContainerStarted","Data":"65c25512a92e3f8a34a3456278c2da01b16d977c5b3789a5d4d73e43048a950e"} Jul 22 02:20:45.047269 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:45.047278 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" event={"ID":"5781ae83-c632-4fcf-8705-3e9dc64dab24","Type":"ContainerStarted","Data":"d4f496c0864337984ef0a49f943211d38e32d28c6c97fb84d023225020933626"} Jul 22 02:20:45.047798 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:45.047290 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" event={"ID":"5781ae83-c632-4fcf-8705-3e9dc64dab24","Type":"ContainerStarted","Data":"410f49de81a965e08e44c7372b34767153011c85007777f73eee3c818440a2ac"} Jul 22 02:20:45.070878 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:45.070810 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5dfbd58f-45jnn" podStartSLOduration=1.756614639 podStartE2EDuration="10.070795913s" totalImagesPullingTime="8.314181274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:45.068908435 +0000 UTC m=+75.977773437" watchObservedRunningTime="2026-07-22 02:20:45.070795913 +0000 UTC m=+75.979660886" Jul 22 02:20:45.331021 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:45.330940 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:20:56.714370 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.714253 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qznvs"] Jul 22 02:20:56.721176 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.721152 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.723964 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.723942 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 02:20:56.724789 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.724769 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 02:20:56.725085 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.725067 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 02:20:56.725167 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.725075 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 02:20:56.725167 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.725110 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 02:20:56.725167 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.725133 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 02:20:56.728888 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.728871 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-d62vw\"" Jul 22 02:20:56.795546 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.795514 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e4416035-20a9-41e2-a938-9dbaf8c68f20-node-exporter-textfile\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.795713 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.795555 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e4416035-20a9-41e2-a938-9dbaf8c68f20-node-exporter-tls\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.795713 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.795573 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bfrb\" (UniqueName: \"kubernetes.io/projected/e4416035-20a9-41e2-a938-9dbaf8c68f20-kube-api-access-6bfrb\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.795713 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.795591 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4416035-20a9-41e2-a938-9dbaf8c68f20-metrics-client-ca\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.795713 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.795689 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e4416035-20a9-41e2-a938-9dbaf8c68f20-root\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.795898 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.795722 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e4416035-20a9-41e2-a938-9dbaf8c68f20-node-exporter-wtmp\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.795898 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.795778 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4416035-20a9-41e2-a938-9dbaf8c68f20-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.795898 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.795844 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e4416035-20a9-41e2-a938-9dbaf8c68f20-node-exporter-accelerators-collector-config\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.795898 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.795862 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4416035-20a9-41e2-a938-9dbaf8c68f20-sys\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.896491 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.896455 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e4416035-20a9-41e2-a938-9dbaf8c68f20-root\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.896491 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.896490 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e4416035-20a9-41e2-a938-9dbaf8c68f20-node-exporter-wtmp\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.896720 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.896517 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4416035-20a9-41e2-a938-9dbaf8c68f20-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.896720 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.896584 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e4416035-20a9-41e2-a938-9dbaf8c68f20-root\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.896720 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.896642 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e4416035-20a9-41e2-a938-9dbaf8c68f20-node-exporter-accelerators-collector-config\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.896720 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.896669 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4416035-20a9-41e2-a938-9dbaf8c68f20-sys\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.896720 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.896644 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e4416035-20a9-41e2-a938-9dbaf8c68f20-node-exporter-wtmp\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.896996 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.896699 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e4416035-20a9-41e2-a938-9dbaf8c68f20-node-exporter-textfile\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.896996 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.896744 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4416035-20a9-41e2-a938-9dbaf8c68f20-sys\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.896996 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.896793 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e4416035-20a9-41e2-a938-9dbaf8c68f20-node-exporter-tls\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.896996 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.896846 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bfrb\" (UniqueName: \"kubernetes.io/projected/e4416035-20a9-41e2-a938-9dbaf8c68f20-kube-api-access-6bfrb\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.896996 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.896878 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4416035-20a9-41e2-a938-9dbaf8c68f20-metrics-client-ca\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.897209 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.897057 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e4416035-20a9-41e2-a938-9dbaf8c68f20-node-exporter-textfile\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.897264 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.897245 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e4416035-20a9-41e2-a938-9dbaf8c68f20-node-exporter-accelerators-collector-config\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.897585 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.897566 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4416035-20a9-41e2-a938-9dbaf8c68f20-metrics-client-ca\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.899056 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.899039 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4416035-20a9-41e2-a938-9dbaf8c68f20-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.899201 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.899185 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e4416035-20a9-41e2-a938-9dbaf8c68f20-node-exporter-tls\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.905773 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.905752 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bfrb\" (UniqueName: \"kubernetes.io/projected/e4416035-20a9-41e2-a938-9dbaf8c68f20-kube-api-access-6bfrb\") pod \"node-exporter-qznvs\" (UID: \"e4416035-20a9-41e2-a938-9dbaf8c68f20\") " pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:56.995865 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:56.995782 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-68dcf7c957-6cgls" Jul 22 02:20:57.030506 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:57.030482 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qznvs" Jul 22 02:20:58.197505 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:20:58.197473 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4416035_20a9_41e2_a938_9dbaf8c68f20.slice/crio-099444befa8d82ef0a515424ab6848dbcce6ee7e1a4f2cd47d782f0c8dbce602 WatchSource:0}: Error finding container 099444befa8d82ef0a515424ab6848dbcce6ee7e1a4f2cd47d782f0c8dbce602: Status 404 returned error can't find the container with id 099444befa8d82ef0a515424ab6848dbcce6ee7e1a4f2cd47d782f0c8dbce602 Jul 22 02:20:59.096238 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:59.096198 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qznvs" event={"ID":"e4416035-20a9-41e2-a938-9dbaf8c68f20","Type":"ContainerStarted","Data":"099444befa8d82ef0a515424ab6848dbcce6ee7e1a4f2cd47d782f0c8dbce602"} Jul 22 02:20:59.097691 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:59.097666 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt" event={"ID":"446feb28-20e9-4ab6-a1c4-f03df2cde5e4","Type":"ContainerStarted","Data":"a40131dc4d5ccbb09534f91cd3ab806d8d2ccbff12a9d81c2af84056bd508c39"} Jul 22 02:20:59.097934 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:59.097919 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt" Jul 22 02:20:59.099659 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:59.099641 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt" Jul 22 02:20:59.114611 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:20:59.114571 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6db6f98976-b9ppt" podStartSLOduration=1.67919478 podStartE2EDuration="24.114557027s" totalImagesPullingTime="22.435362247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:59.113809549 +0000 UTC m=+90.022674541" watchObservedRunningTime="2026-07-22 02:20:59.114557027 +0000 UTC m=+90.023422021" Jul 22 02:21:00.104160 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.104122 2557 generic.go:356] "Generic (PLEG): container finished" podID="e4416035-20a9-41e2-a938-9dbaf8c68f20" containerID="2705d3f79341b5566d9b17cd617e19aec4e9df1ecf317430f092b76ca19f2181" exitCode=0 Jul 22 02:21:00.104576 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.104211 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qznvs" event={"ID":"e4416035-20a9-41e2-a938-9dbaf8c68f20","Type":"ContainerDied","Data":"2705d3f79341b5566d9b17cd617e19aec4e9df1ecf317430f092b76ca19f2181"} Jul 22 02:21:00.349871 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.349788 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" podUID="b6e35b92-2a47-4594-8b5e-9e8110892637" containerName="registry" containerID="cri-o://1e5aff9999412c965e9a54d2fe13123a9f630480e227e9ee8f8d48646b0577c1" gracePeriod=30 Jul 22 02:21:00.604990 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.604958 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:21:00.731305 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.731278 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-certificates\") pod \"b6e35b92-2a47-4594-8b5e-9e8110892637\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " Jul 22 02:21:00.731462 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.731316 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls\") pod \"b6e35b92-2a47-4594-8b5e-9e8110892637\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " Jul 22 02:21:00.731462 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.731358 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6e35b92-2a47-4594-8b5e-9e8110892637-installation-pull-secrets\") pod \"b6e35b92-2a47-4594-8b5e-9e8110892637\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " Jul 22 02:21:00.731462 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.731383 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-bound-sa-token\") pod \"b6e35b92-2a47-4594-8b5e-9e8110892637\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " Jul 22 02:21:00.731462 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.731415 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6e35b92-2a47-4594-8b5e-9e8110892637-ca-trust-extracted\") pod \"b6e35b92-2a47-4594-8b5e-9e8110892637\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " Jul 22 02:21:00.731669 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.731482 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6e35b92-2a47-4594-8b5e-9e8110892637-image-registry-private-configuration\") pod \"b6e35b92-2a47-4594-8b5e-9e8110892637\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " Jul 22 02:21:00.731669 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.731513 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pz5tr\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-kube-api-access-pz5tr\") pod \"b6e35b92-2a47-4594-8b5e-9e8110892637\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " Jul 22 02:21:00.731765 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.731717 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b6e35b92-2a47-4594-8b5e-9e8110892637" (UID: "b6e35b92-2a47-4594-8b5e-9e8110892637"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:21:00.731925 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.731545 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6e35b92-2a47-4594-8b5e-9e8110892637-trusted-ca\") pod \"b6e35b92-2a47-4594-8b5e-9e8110892637\" (UID: \"b6e35b92-2a47-4594-8b5e-9e8110892637\") " Jul 22 02:21:00.732232 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.732168 2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-certificates\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:21:00.732232 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.732212 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6e35b92-2a47-4594-8b5e-9e8110892637-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b6e35b92-2a47-4594-8b5e-9e8110892637" (UID: "b6e35b92-2a47-4594-8b5e-9e8110892637"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:21:00.734465 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.734422 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6e35b92-2a47-4594-8b5e-9e8110892637-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b6e35b92-2a47-4594-8b5e-9e8110892637" (UID: "b6e35b92-2a47-4594-8b5e-9e8110892637"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:21:00.734572 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.734520 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b6e35b92-2a47-4594-8b5e-9e8110892637" (UID: "b6e35b92-2a47-4594-8b5e-9e8110892637"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:21:00.734572 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.734528 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6e35b92-2a47-4594-8b5e-9e8110892637-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b6e35b92-2a47-4594-8b5e-9e8110892637" (UID: "b6e35b92-2a47-4594-8b5e-9e8110892637"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:21:00.734654 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.734585 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b6e35b92-2a47-4594-8b5e-9e8110892637" (UID: "b6e35b92-2a47-4594-8b5e-9e8110892637"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:21:00.734719 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.734667 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-kube-api-access-pz5tr" (OuterVolumeSpecName: "kube-api-access-pz5tr") pod "b6e35b92-2a47-4594-8b5e-9e8110892637" (UID: "b6e35b92-2a47-4594-8b5e-9e8110892637"). InnerVolumeSpecName "kube-api-access-pz5tr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:21:00.740447 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.740425 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6e35b92-2a47-4594-8b5e-9e8110892637-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b6e35b92-2a47-4594-8b5e-9e8110892637" (UID: "b6e35b92-2a47-4594-8b5e-9e8110892637"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:21:00.832619 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.832591 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-registry-tls\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:21:00.832619 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.832616 2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6e35b92-2a47-4594-8b5e-9e8110892637-installation-pull-secrets\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:21:00.832773 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.832628 2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-bound-sa-token\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:21:00.832773 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.832636 2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6e35b92-2a47-4594-8b5e-9e8110892637-ca-trust-extracted\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:21:00.832773 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.832645 2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6e35b92-2a47-4594-8b5e-9e8110892637-image-registry-private-configuration\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:21:00.832773 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.832654 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pz5tr\" (UniqueName: \"kubernetes.io/projected/b6e35b92-2a47-4594-8b5e-9e8110892637-kube-api-access-pz5tr\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:21:00.832773 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:00.832663 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6e35b92-2a47-4594-8b5e-9e8110892637-trusted-ca\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:21:01.115805 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:01.115724 2557 generic.go:356] "Generic (PLEG): container finished" podID="b6e35b92-2a47-4594-8b5e-9e8110892637" containerID="1e5aff9999412c965e9a54d2fe13123a9f630480e227e9ee8f8d48646b0577c1" exitCode=0 Jul 22 02:21:01.115805 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:01.115789 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" Jul 22 02:21:01.116279 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:01.115808 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" event={"ID":"b6e35b92-2a47-4594-8b5e-9e8110892637","Type":"ContainerDied","Data":"1e5aff9999412c965e9a54d2fe13123a9f630480e227e9ee8f8d48646b0577c1"} Jul 22 02:21:01.116279 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:01.115917 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84c6f8455b-2hn2c" event={"ID":"b6e35b92-2a47-4594-8b5e-9e8110892637","Type":"ContainerDied","Data":"fb9eaa74156babca9f747ea626735a494d4bd2573d6f55773b48cb11b06d0147"} Jul 22 02:21:01.116279 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:01.115945 2557 scope.go:117] "RemoveContainer" containerID="1e5aff9999412c965e9a54d2fe13123a9f630480e227e9ee8f8d48646b0577c1" Jul 22 02:21:01.118729 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:01.118705 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qznvs" event={"ID":"e4416035-20a9-41e2-a938-9dbaf8c68f20","Type":"ContainerStarted","Data":"0fc03d3f6730357ff000fa426afeb1a01eac9dc7ec753ffd2e778fb5183e4c36"} Jul 22 02:21:01.118893 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:01.118753 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qznvs" event={"ID":"e4416035-20a9-41e2-a938-9dbaf8c68f20","Type":"ContainerStarted","Data":"7170c6549c22315ab4c524b3f15d5f6ce3ea8944160de6fec4fae5aafcc48bfb"} Jul 22 02:21:01.130834 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:01.130796 2557 scope.go:117] "RemoveContainer" containerID="1e5aff9999412c965e9a54d2fe13123a9f630480e227e9ee8f8d48646b0577c1" Jul 22 02:21:01.131122 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:21:01.131101 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e5aff9999412c965e9a54d2fe13123a9f630480e227e9ee8f8d48646b0577c1\": container with ID starting with 1e5aff9999412c965e9a54d2fe13123a9f630480e227e9ee8f8d48646b0577c1 not found: ID does not exist" containerID="1e5aff9999412c965e9a54d2fe13123a9f630480e227e9ee8f8d48646b0577c1" Jul 22 02:21:01.131192 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:01.131130 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e5aff9999412c965e9a54d2fe13123a9f630480e227e9ee8f8d48646b0577c1"} err="failed to get container status \"1e5aff9999412c965e9a54d2fe13123a9f630480e227e9ee8f8d48646b0577c1\": rpc error: code = NotFound desc = could not find container \"1e5aff9999412c965e9a54d2fe13123a9f630480e227e9ee8f8d48646b0577c1\": container with ID starting with 1e5aff9999412c965e9a54d2fe13123a9f630480e227e9ee8f8d48646b0577c1 not found: ID does not exist" Jul 22 02:21:01.143772 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:01.143721 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qznvs" podStartSLOduration=4.234769499 podStartE2EDuration="5.143708047s" totalImagesPullingTime="839.395101ms" totalInitContainerRuntime="69.543447ms" isStatefulPod=true podCreationTimestamp="2026-07-22 02:20:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:01.143433153 +0000 UTC m=+92.052298149" watchObservedRunningTime="2026-07-22 02:21:01.143708047 +0000 UTC m=+92.052573054" Jul 22 02:21:01.164234 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:01.164211 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84c6f8455b-2hn2c"] Jul 22 02:21:01.169457 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:01.169437 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-84c6f8455b-2hn2c"] Jul 22 02:21:01.607873 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:01.607840 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6e35b92-2a47-4594-8b5e-9e8110892637" path="/var/lib/kubelet/pods/b6e35b92-2a47-4594-8b5e-9e8110892637/volumes" Jul 22 02:21:15.050227 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:21:15.050191 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tqrxj" Jul 22 02:24:29.501682 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:24:29.501654 2557 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 02:25:25.641493 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:25.641416 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sm7br"] Jul 22 02:25:25.641903 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:25.641650 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6e35b92-2a47-4594-8b5e-9e8110892637" containerName="registry" Jul 22 02:25:25.641903 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:25.641660 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6e35b92-2a47-4594-8b5e-9e8110892637" containerName="registry" Jul 22 02:25:25.641903 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:25.641705 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6e35b92-2a47-4594-8b5e-9e8110892637" containerName="registry" Jul 22 02:25:25.643557 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:25.643541 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sm7br" Jul 22 02:25:25.646704 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:25.646686 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:25:25.646785 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:25.646746 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 22 02:25:25.646785 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:25.646744 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-rzscl\"" Jul 22 02:25:25.654155 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:25.654135 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sm7br"] Jul 22 02:25:25.780944 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:25.780911 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/168cf675-4e4b-4f49-9c23-f63150dbcd19-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sm7br\" (UID: \"168cf675-4e4b-4f49-9c23-f63150dbcd19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sm7br" Jul 22 02:25:25.781136 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:25.780959 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8cqh\" (UniqueName: \"kubernetes.io/projected/168cf675-4e4b-4f49-9c23-f63150dbcd19-kube-api-access-g8cqh\") pod \"openshift-lws-operator-fd8ccff4c-sm7br\" (UID: \"168cf675-4e4b-4f49-9c23-f63150dbcd19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sm7br" Jul 22 02:25:25.882213 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:25.882174 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8cqh\" (UniqueName: \"kubernetes.io/projected/168cf675-4e4b-4f49-9c23-f63150dbcd19-kube-api-access-g8cqh\") pod \"openshift-lws-operator-fd8ccff4c-sm7br\" (UID: \"168cf675-4e4b-4f49-9c23-f63150dbcd19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sm7br" Jul 22 02:25:25.882338 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:25.882266 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/168cf675-4e4b-4f49-9c23-f63150dbcd19-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sm7br\" (UID: \"168cf675-4e4b-4f49-9c23-f63150dbcd19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sm7br" Jul 22 02:25:25.882681 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:25.882662 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/168cf675-4e4b-4f49-9c23-f63150dbcd19-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sm7br\" (UID: \"168cf675-4e4b-4f49-9c23-f63150dbcd19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sm7br" Jul 22 02:25:25.891001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:25.890971 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8cqh\" (UniqueName: \"kubernetes.io/projected/168cf675-4e4b-4f49-9c23-f63150dbcd19-kube-api-access-g8cqh\") pod \"openshift-lws-operator-fd8ccff4c-sm7br\" (UID: \"168cf675-4e4b-4f49-9c23-f63150dbcd19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sm7br" Jul 22 02:25:25.952109 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:25.952025 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sm7br" Jul 22 02:25:26.086111 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:26.085974 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sm7br"] Jul 22 02:25:26.090676 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:25:26.090644 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod168cf675_4e4b_4f49_9c23_f63150dbcd19.slice/crio-473c684858c4a6c7b4cd3f3f6a42bc024c5e4a5f7ddd4a4db38ec0c47f4c58fa WatchSource:0}: Error finding container 473c684858c4a6c7b4cd3f3f6a42bc024c5e4a5f7ddd4a4db38ec0c47f4c58fa: Status 404 returned error can't find the container with id 473c684858c4a6c7b4cd3f3f6a42bc024c5e4a5f7ddd4a4db38ec0c47f4c58fa Jul 22 02:25:26.091849 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:26.091805 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:25:27.000036 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:26.999999 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sm7br" event={"ID":"168cf675-4e4b-4f49-9c23-f63150dbcd19","Type":"ContainerStarted","Data":"473c684858c4a6c7b4cd3f3f6a42bc024c5e4a5f7ddd4a4db38ec0c47f4c58fa"} Jul 22 02:25:30.011116 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:30.011078 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sm7br" event={"ID":"168cf675-4e4b-4f49-9c23-f63150dbcd19","Type":"ContainerStarted","Data":"fcb6de56db45b4778bb547a7ef45fccab060b0d279df7c6dd55ce7aabb616c73"} Jul 22 02:25:30.035734 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:30.035686 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sm7br" podStartSLOduration=1.9475158879999999 podStartE2EDuration="5.035671836s" totalImagesPullingTime="3.088155948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:25:30.035198802 +0000 UTC m=+360.944063800" watchObservedRunningTime="2026-07-22 02:25:30.035671836 +0000 UTC m=+360.944536833" Jul 22 02:25:45.668465 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.668429 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8"] Jul 22 02:25:45.670598 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.670582 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:45.674157 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.674135 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 02:25:45.674277 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.674135 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gf5lf\"" Jul 22 02:25:45.674398 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.674382 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 02:25:45.674654 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.674639 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 02:25:45.674707 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.674684 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 02:25:45.686348 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.686330 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8"] Jul 22 02:25:45.832583 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.832543 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e8e68259-7a67-470c-b249-733ac647e978-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-xr8f8\" (UID: \"e8e68259-7a67-470c-b249-733ac647e978\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:45.832583 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.832587 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e8e68259-7a67-470c-b249-733ac647e978-manifests\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-xr8f8\" (UID: \"e8e68259-7a67-470c-b249-733ac647e978\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:45.832801 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.832669 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8e68259-7a67-470c-b249-733ac647e978-tmp\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-xr8f8\" (UID: \"e8e68259-7a67-470c-b249-733ac647e978\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:45.832801 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.832704 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffsb7\" (UniqueName: \"kubernetes.io/projected/e8e68259-7a67-470c-b249-733ac647e978-kube-api-access-ffsb7\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-xr8f8\" (UID: \"e8e68259-7a67-470c-b249-733ac647e978\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:45.832801 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.832732 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e8e68259-7a67-470c-b249-733ac647e978-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-xr8f8\" (UID: \"e8e68259-7a67-470c-b249-733ac647e978\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:45.933534 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.933454 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8e68259-7a67-470c-b249-733ac647e978-tmp\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-xr8f8\" (UID: \"e8e68259-7a67-470c-b249-733ac647e978\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:45.933534 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.933487 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffsb7\" (UniqueName: \"kubernetes.io/projected/e8e68259-7a67-470c-b249-733ac647e978-kube-api-access-ffsb7\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-xr8f8\" (UID: \"e8e68259-7a67-470c-b249-733ac647e978\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:45.933534 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.933511 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e8e68259-7a67-470c-b249-733ac647e978-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-xr8f8\" (UID: \"e8e68259-7a67-470c-b249-733ac647e978\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:45.933741 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.933549 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e8e68259-7a67-470c-b249-733ac647e978-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-xr8f8\" (UID: \"e8e68259-7a67-470c-b249-733ac647e978\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:45.933741 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.933586 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e8e68259-7a67-470c-b249-733ac647e978-manifests\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-xr8f8\" (UID: \"e8e68259-7a67-470c-b249-733ac647e978\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:45.934005 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.933978 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e8e68259-7a67-470c-b249-733ac647e978-manifests\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-xr8f8\" (UID: \"e8e68259-7a67-470c-b249-733ac647e978\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:45.935916 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.935883 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8e68259-7a67-470c-b249-733ac647e978-tmp\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-xr8f8\" (UID: \"e8e68259-7a67-470c-b249-733ac647e978\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:45.936025 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.935995 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e8e68259-7a67-470c-b249-733ac647e978-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-xr8f8\" (UID: \"e8e68259-7a67-470c-b249-733ac647e978\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:45.936091 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.936073 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e8e68259-7a67-470c-b249-733ac647e978-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-xr8f8\" (UID: \"e8e68259-7a67-470c-b249-733ac647e978\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:45.942692 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.942669 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffsb7\" (UniqueName: \"kubernetes.io/projected/e8e68259-7a67-470c-b249-733ac647e978-kube-api-access-ffsb7\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-xr8f8\" (UID: \"e8e68259-7a67-470c-b249-733ac647e978\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:45.980816 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:45.980782 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:46.114905 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:46.114880 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8"] Jul 22 02:25:46.117217 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:25:46.117185 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8e68259_7a67_470c_b249_733ac647e978.slice/crio-c396fffb53c1f15fe11b24b6e7d4525da8189d6cbea9488afd96cfc5febeecbd WatchSource:0}: Error finding container c396fffb53c1f15fe11b24b6e7d4525da8189d6cbea9488afd96cfc5febeecbd: Status 404 returned error can't find the container with id c396fffb53c1f15fe11b24b6e7d4525da8189d6cbea9488afd96cfc5febeecbd Jul 22 02:25:47.072157 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:47.072109 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" event={"ID":"e8e68259-7a67-470c-b249-733ac647e978","Type":"ContainerStarted","Data":"c396fffb53c1f15fe11b24b6e7d4525da8189d6cbea9488afd96cfc5febeecbd"} Jul 22 02:25:50.085153 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:50.085115 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" event={"ID":"e8e68259-7a67-470c-b249-733ac647e978","Type":"ContainerStarted","Data":"1b60332a82feab44f4f62150918918847eeca0d9863e381ae154d744ad12510a"} Jul 22 02:25:51.090359 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:51.090321 2557 generic.go:356] "Generic (PLEG): container finished" podID="e8e68259-7a67-470c-b249-733ac647e978" containerID="1b60332a82feab44f4f62150918918847eeca0d9863e381ae154d744ad12510a" exitCode=0 Jul 22 02:25:51.090763 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:51.090372 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" event={"ID":"e8e68259-7a67-470c-b249-733ac647e978","Type":"ContainerDied","Data":"1b60332a82feab44f4f62150918918847eeca0d9863e381ae154d744ad12510a"} Jul 22 02:25:52.095191 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:52.095160 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" event={"ID":"e8e68259-7a67-470c-b249-733ac647e978","Type":"ContainerStarted","Data":"9cba3294d0a4068d42a1c32db50ff628fd1856c51d96c658084e3924a5375694"} Jul 22 02:25:52.095557 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:52.095284 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:25:52.119950 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:25:52.119910 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" podStartSLOduration=2.8037807089999998 podStartE2EDuration="7.119895038s" totalImagesPullingTime="3.251417173s" totalInitContainerRuntime="1.064697156s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:25:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:25:52.118703858 +0000 UTC m=+383.027568853" watchObservedRunningTime="2026-07-22 02:25:52.119895038 +0000 UTC m=+383.028760034" Jul 22 02:26:03.101094 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:03.101062 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-xr8f8" Jul 22 02:26:06.672233 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.672203 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-65c4bb8965-gt6rw"] Jul 22 02:26:06.682978 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.682956 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65c4bb8965-gt6rw" Jul 22 02:26:06.692333 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.692313 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 22 02:26:06.692451 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.692432 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 02:26:06.692493 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.692432 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 02:26:06.692493 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.692432 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hlsdx\"" Jul 22 02:26:06.693520 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.693505 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 22 02:26:06.695859 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.695815 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65c4bb8965-gt6rw"] Jul 22 02:26:06.696370 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.696351 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b35918df-d4bf-4130-8e89-9a9b9ca4eb95-tmp\") pod \"kube-auth-proxy-65c4bb8965-gt6rw\" (UID: \"b35918df-d4bf-4130-8e89-9a9b9ca4eb95\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-gt6rw" Jul 22 02:26:06.696453 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.696383 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4grnc\" (UniqueName: \"kubernetes.io/projected/b35918df-d4bf-4130-8e89-9a9b9ca4eb95-kube-api-access-4grnc\") pod \"kube-auth-proxy-65c4bb8965-gt6rw\" (UID: \"b35918df-d4bf-4130-8e89-9a9b9ca4eb95\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-gt6rw" Jul 22 02:26:06.696714 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.696540 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b35918df-d4bf-4130-8e89-9a9b9ca4eb95-tls-certs\") pod \"kube-auth-proxy-65c4bb8965-gt6rw\" (UID: \"b35918df-d4bf-4130-8e89-9a9b9ca4eb95\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-gt6rw" Jul 22 02:26:06.797105 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.797076 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b35918df-d4bf-4130-8e89-9a9b9ca4eb95-tls-certs\") pod \"kube-auth-proxy-65c4bb8965-gt6rw\" (UID: \"b35918df-d4bf-4130-8e89-9a9b9ca4eb95\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-gt6rw" Jul 22 02:26:06.797278 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.797139 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b35918df-d4bf-4130-8e89-9a9b9ca4eb95-tmp\") pod \"kube-auth-proxy-65c4bb8965-gt6rw\" (UID: \"b35918df-d4bf-4130-8e89-9a9b9ca4eb95\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-gt6rw" Jul 22 02:26:06.797278 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.797159 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4grnc\" (UniqueName: \"kubernetes.io/projected/b35918df-d4bf-4130-8e89-9a9b9ca4eb95-kube-api-access-4grnc\") pod \"kube-auth-proxy-65c4bb8965-gt6rw\" (UID: \"b35918df-d4bf-4130-8e89-9a9b9ca4eb95\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-gt6rw" Jul 22 02:26:06.799463 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.799439 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b35918df-d4bf-4130-8e89-9a9b9ca4eb95-tmp\") pod \"kube-auth-proxy-65c4bb8965-gt6rw\" (UID: \"b35918df-d4bf-4130-8e89-9a9b9ca4eb95\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-gt6rw" Jul 22 02:26:06.799585 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.799566 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b35918df-d4bf-4130-8e89-9a9b9ca4eb95-tls-certs\") pod \"kube-auth-proxy-65c4bb8965-gt6rw\" (UID: \"b35918df-d4bf-4130-8e89-9a9b9ca4eb95\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-gt6rw" Jul 22 02:26:06.839647 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.839621 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4grnc\" (UniqueName: \"kubernetes.io/projected/b35918df-d4bf-4130-8e89-9a9b9ca4eb95-kube-api-access-4grnc\") pod \"kube-auth-proxy-65c4bb8965-gt6rw\" (UID: \"b35918df-d4bf-4130-8e89-9a9b9ca4eb95\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-gt6rw" Jul 22 02:26:06.875808 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.875781 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wfh7b"] Jul 22 02:26:06.877810 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.877796 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-wfh7b" Jul 22 02:26:06.884037 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.884015 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-rvq9s\"" Jul 22 02:26:06.884894 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.884521 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 22 02:26:06.897780 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.897761 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42c5d187-3867-4a46-8f7c-980f572dc26a-cert\") pod \"odh-model-controller-7998c67dc4-wfh7b\" (UID: \"42c5d187-3867-4a46-8f7c-980f572dc26a\") " pod="opendatahub/odh-model-controller-7998c67dc4-wfh7b" Jul 22 02:26:06.897905 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.897809 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r8j7\" (UniqueName: \"kubernetes.io/projected/42c5d187-3867-4a46-8f7c-980f572dc26a-kube-api-access-2r8j7\") pod \"odh-model-controller-7998c67dc4-wfh7b\" (UID: \"42c5d187-3867-4a46-8f7c-980f572dc26a\") " pod="opendatahub/odh-model-controller-7998c67dc4-wfh7b" Jul 22 02:26:06.906212 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.906191 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wfh7b"] Jul 22 02:26:06.991663 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.991640 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65c4bb8965-gt6rw" Jul 22 02:26:06.998360 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.998336 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42c5d187-3867-4a46-8f7c-980f572dc26a-cert\") pod \"odh-model-controller-7998c67dc4-wfh7b\" (UID: \"42c5d187-3867-4a46-8f7c-980f572dc26a\") " pod="opendatahub/odh-model-controller-7998c67dc4-wfh7b" Jul 22 02:26:06.998488 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:06.998403 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r8j7\" (UniqueName: \"kubernetes.io/projected/42c5d187-3867-4a46-8f7c-980f572dc26a-kube-api-access-2r8j7\") pod \"odh-model-controller-7998c67dc4-wfh7b\" (UID: \"42c5d187-3867-4a46-8f7c-980f572dc26a\") " pod="opendatahub/odh-model-controller-7998c67dc4-wfh7b" Jul 22 02:26:06.998551 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:26:06.998500 2557 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 22 02:26:06.998598 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:26:06.998571 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42c5d187-3867-4a46-8f7c-980f572dc26a-cert podName:42c5d187-3867-4a46-8f7c-980f572dc26a nodeName:}" failed. No retries permitted until 2026-07-22 02:26:07.498550527 +0000 UTC m=+398.407415507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/42c5d187-3867-4a46-8f7c-980f572dc26a-cert") pod "odh-model-controller-7998c67dc4-wfh7b" (UID: "42c5d187-3867-4a46-8f7c-980f572dc26a") : secret "odh-model-controller-webhook-cert" not found Jul 22 02:26:07.021801 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:07.021773 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r8j7\" (UniqueName: \"kubernetes.io/projected/42c5d187-3867-4a46-8f7c-980f572dc26a-kube-api-access-2r8j7\") pod \"odh-model-controller-7998c67dc4-wfh7b\" (UID: \"42c5d187-3867-4a46-8f7c-980f572dc26a\") " pod="opendatahub/odh-model-controller-7998c67dc4-wfh7b" Jul 22 02:26:07.142128 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:07.140256 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65c4bb8965-gt6rw"] Jul 22 02:26:07.150231 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:07.150200 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65c4bb8965-gt6rw" event={"ID":"b35918df-d4bf-4130-8e89-9a9b9ca4eb95","Type":"ContainerStarted","Data":"69987fa442bdd77a49a4611b572cba97c53c3481d5de523906e1da0435d5a6a0"} Jul 22 02:26:07.502438 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:07.502399 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42c5d187-3867-4a46-8f7c-980f572dc26a-cert\") pod \"odh-model-controller-7998c67dc4-wfh7b\" (UID: \"42c5d187-3867-4a46-8f7c-980f572dc26a\") " pod="opendatahub/odh-model-controller-7998c67dc4-wfh7b" Jul 22 02:26:07.502618 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:26:07.502532 2557 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 22 02:26:07.502618 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:26:07.502596 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42c5d187-3867-4a46-8f7c-980f572dc26a-cert podName:42c5d187-3867-4a46-8f7c-980f572dc26a nodeName:}" failed. No retries permitted until 2026-07-22 02:26:08.502582073 +0000 UTC m=+399.411447046 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/42c5d187-3867-4a46-8f7c-980f572dc26a-cert") pod "odh-model-controller-7998c67dc4-wfh7b" (UID: "42c5d187-3867-4a46-8f7c-980f572dc26a") : secret "odh-model-controller-webhook-cert" not found Jul 22 02:26:08.121813 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.121782 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bqxlc"] Jul 22 02:26:08.124303 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.124284 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-bqxlc" Jul 22 02:26:08.128876 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.128850 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 22 02:26:08.131780 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.131725 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-cvpgg\"" Jul 22 02:26:08.153499 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.153441 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bqxlc"] Jul 22 02:26:08.208172 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.208135 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sjnx\" (UniqueName: \"kubernetes.io/projected/ef9503af-c09f-4cfc-a58d-b459e17ff907-kube-api-access-2sjnx\") pod \"model-serving-api-54d89bd5fc-bqxlc\" (UID: \"ef9503af-c09f-4cfc-a58d-b459e17ff907\") " pod="opendatahub/model-serving-api-54d89bd5fc-bqxlc" Jul 22 02:26:08.208313 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.208222 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef9503af-c09f-4cfc-a58d-b459e17ff907-tls-certs\") pod \"model-serving-api-54d89bd5fc-bqxlc\" (UID: \"ef9503af-c09f-4cfc-a58d-b459e17ff907\") " pod="opendatahub/model-serving-api-54d89bd5fc-bqxlc" Jul 22 02:26:08.308915 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.308879 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sjnx\" (UniqueName: \"kubernetes.io/projected/ef9503af-c09f-4cfc-a58d-b459e17ff907-kube-api-access-2sjnx\") pod \"model-serving-api-54d89bd5fc-bqxlc\" (UID: \"ef9503af-c09f-4cfc-a58d-b459e17ff907\") " pod="opendatahub/model-serving-api-54d89bd5fc-bqxlc" Jul 22 02:26:08.309102 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.308970 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef9503af-c09f-4cfc-a58d-b459e17ff907-tls-certs\") pod \"model-serving-api-54d89bd5fc-bqxlc\" (UID: \"ef9503af-c09f-4cfc-a58d-b459e17ff907\") " pod="opendatahub/model-serving-api-54d89bd5fc-bqxlc" Jul 22 02:26:08.309161 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:26:08.309100 2557 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 22 02:26:08.309213 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:26:08.309163 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef9503af-c09f-4cfc-a58d-b459e17ff907-tls-certs podName:ef9503af-c09f-4cfc-a58d-b459e17ff907 nodeName:}" failed. No retries permitted until 2026-07-22 02:26:08.809143775 +0000 UTC m=+399.718008748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ef9503af-c09f-4cfc-a58d-b459e17ff907-tls-certs") pod "model-serving-api-54d89bd5fc-bqxlc" (UID: "ef9503af-c09f-4cfc-a58d-b459e17ff907") : secret "model-serving-api-tls" not found Jul 22 02:26:08.319116 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.319083 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sjnx\" (UniqueName: \"kubernetes.io/projected/ef9503af-c09f-4cfc-a58d-b459e17ff907-kube-api-access-2sjnx\") pod \"model-serving-api-54d89bd5fc-bqxlc\" (UID: \"ef9503af-c09f-4cfc-a58d-b459e17ff907\") " pod="opendatahub/model-serving-api-54d89bd5fc-bqxlc" Jul 22 02:26:08.511058 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.511021 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42c5d187-3867-4a46-8f7c-980f572dc26a-cert\") pod \"odh-model-controller-7998c67dc4-wfh7b\" (UID: \"42c5d187-3867-4a46-8f7c-980f572dc26a\") " pod="opendatahub/odh-model-controller-7998c67dc4-wfh7b" Jul 22 02:26:08.513750 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.513728 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42c5d187-3867-4a46-8f7c-980f572dc26a-cert\") pod \"odh-model-controller-7998c67dc4-wfh7b\" (UID: \"42c5d187-3867-4a46-8f7c-980f572dc26a\") " pod="opendatahub/odh-model-controller-7998c67dc4-wfh7b" Jul 22 02:26:08.621267 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.621224 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j"] Jul 22 02:26:08.623501 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.623478 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.626516 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.626492 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 22 02:26:08.626657 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.626546 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 22 02:26:08.626852 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.626782 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 22 02:26:08.626967 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.626953 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 02:26:08.627091 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.627074 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-m7w55\"" Jul 22 02:26:08.639815 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.639796 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j"] Jul 22 02:26:08.688783 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.688763 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-wfh7b" Jul 22 02:26:08.712838 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.712798 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.712956 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.712858 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.712956 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.712878 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vngxq\" (UniqueName: \"kubernetes.io/projected/d1cd3d62-ed36-4952-ac62-89ca6313b20d-kube-api-access-vngxq\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.712956 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.712912 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.713081 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.713004 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-token\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.713115 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.713078 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1cd3d62-ed36-4952-ac62-89ca6313b20d-local-certs\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.713160 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.713116 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-cacerts\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.813771 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.813734 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.813953 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.813779 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.813953 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.813805 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vngxq\" (UniqueName: \"kubernetes.io/projected/d1cd3d62-ed36-4952-ac62-89ca6313b20d-kube-api-access-vngxq\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.814065 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.813977 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.814116 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.814075 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef9503af-c09f-4cfc-a58d-b459e17ff907-tls-certs\") pod \"model-serving-api-54d89bd5fc-bqxlc\" (UID: \"ef9503af-c09f-4cfc-a58d-b459e17ff907\") " pod="opendatahub/model-serving-api-54d89bd5fc-bqxlc" Jul 22 02:26:08.814170 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.814121 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-token\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.814170 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.814152 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1cd3d62-ed36-4952-ac62-89ca6313b20d-local-certs\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.814482 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.814455 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.814482 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.814457 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-cacerts\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.817110 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.817063 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.817737 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.817516 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-cacerts\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.817857 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.817740 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.818166 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.818129 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef9503af-c09f-4cfc-a58d-b459e17ff907-tls-certs\") pod \"model-serving-api-54d89bd5fc-bqxlc\" (UID: \"ef9503af-c09f-4cfc-a58d-b459e17ff907\") " pod="opendatahub/model-serving-api-54d89bd5fc-bqxlc" Jul 22 02:26:08.818166 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.818146 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1cd3d62-ed36-4952-ac62-89ca6313b20d-local-certs\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.823310 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.823288 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vngxq\" (UniqueName: \"kubernetes.io/projected/d1cd3d62-ed36-4952-ac62-89ca6313b20d-kube-api-access-vngxq\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.823408 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.823311 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-token\") pod \"istiod-openshift-gateway-7574459656-jmk2j\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:08.829986 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.829963 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wfh7b"] Jul 22 02:26:08.933168 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:08.933135 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:09.036965 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:09.036878 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-bqxlc" Jul 22 02:26:09.409434 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:26:09.409352 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42c5d187_3867_4a46_8f7c_980f572dc26a.slice/crio-3dc0aeb3854c248082bbd1400ad0e6b8e160706b2552fc9a640df5edeb4be597 WatchSource:0}: Error finding container 3dc0aeb3854c248082bbd1400ad0e6b8e160706b2552fc9a640df5edeb4be597: Status 404 returned error can't find the container with id 3dc0aeb3854c248082bbd1400ad0e6b8e160706b2552fc9a640df5edeb4be597 Jul 22 02:26:09.555245 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:09.555217 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bqxlc"] Jul 22 02:26:09.557280 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:26:09.557256 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef9503af_c09f_4cfc_a58d_b459e17ff907.slice/crio-e6762baf56c0dc4d90aba4d61da469cc47f4129d77219af80a9920d664b6bc8b WatchSource:0}: Error finding container e6762baf56c0dc4d90aba4d61da469cc47f4129d77219af80a9920d664b6bc8b: Status 404 returned error can't find the container with id e6762baf56c0dc4d90aba4d61da469cc47f4129d77219af80a9920d664b6bc8b Jul 22 02:26:09.579153 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:09.579134 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j"] Jul 22 02:26:09.581362 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:26:09.581335 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1cd3d62_ed36_4952_ac62_89ca6313b20d.slice/crio-321d0dfdfc7ee71030e721136f6804afeeba099d2aeb4c35a210b4104487abb0 WatchSource:0}: Error finding container 321d0dfdfc7ee71030e721136f6804afeeba099d2aeb4c35a210b4104487abb0: Status 404 returned error can't find the container with id 321d0dfdfc7ee71030e721136f6804afeeba099d2aeb4c35a210b4104487abb0 Jul 22 02:26:10.169245 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:10.169152 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-bqxlc" event={"ID":"ef9503af-c09f-4cfc-a58d-b459e17ff907","Type":"ContainerStarted","Data":"e6762baf56c0dc4d90aba4d61da469cc47f4129d77219af80a9920d664b6bc8b"} Jul 22 02:26:10.170606 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:10.170542 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" event={"ID":"d1cd3d62-ed36-4952-ac62-89ca6313b20d","Type":"ContainerStarted","Data":"321d0dfdfc7ee71030e721136f6804afeeba099d2aeb4c35a210b4104487abb0"} Jul 22 02:26:10.172053 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:10.171992 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-wfh7b" event={"ID":"42c5d187-3867-4a46-8f7c-980f572dc26a","Type":"ContainerStarted","Data":"3dc0aeb3854c248082bbd1400ad0e6b8e160706b2552fc9a640df5edeb4be597"} Jul 22 02:26:10.174619 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:10.173945 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65c4bb8965-gt6rw" event={"ID":"b35918df-d4bf-4130-8e89-9a9b9ca4eb95","Type":"ContainerStarted","Data":"159ff1832b488ff32c626999f2a784b2cba23a489387df37decd2bc9b957ab6f"} Jul 22 02:26:13.190325 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:13.190287 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-bqxlc" event={"ID":"ef9503af-c09f-4cfc-a58d-b459e17ff907","Type":"ContainerStarted","Data":"083646542e54daed63dbeb4cb4fb0d80b83d8be22d0f1aff2e510b8bf6092e62"} Jul 22 02:26:13.190974 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:13.190418 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-bqxlc" Jul 22 02:26:13.191994 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:13.191966 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-wfh7b" event={"ID":"42c5d187-3867-4a46-8f7c-980f572dc26a","Type":"ContainerStarted","Data":"19717f29a97d278de082fb62f27c3e11c1efff6863bf54d77b3724f9b50df79a"} Jul 22 02:26:13.192131 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:13.192114 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-wfh7b" Jul 22 02:26:13.221240 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:13.221198 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-65c4bb8965-gt6rw" podStartSLOduration=4.924891571 podStartE2EDuration="7.221182469s" totalImagesPullingTime="2.296290898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:26:10.19367316 +0000 UTC m=+401.102538156" watchObservedRunningTime="2026-07-22 02:26:13.221182469 +0000 UTC m=+404.130047467" Jul 22 02:26:13.222389 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:13.222351 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-bqxlc" podStartSLOduration=2.323099496 podStartE2EDuration="5.222337401s" totalImagesPullingTime="2.899237905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:26:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:26:13.220672269 +0000 UTC m=+404.129537263" watchObservedRunningTime="2026-07-22 02:26:13.222337401 +0000 UTC m=+404.131202397" Jul 22 02:26:13.240755 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:13.240714 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-wfh7b" podStartSLOduration=4.188197321 podStartE2EDuration="7.240701898s" totalImagesPullingTime="3.052504577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:26:13.240130828 +0000 UTC m=+404.148995824" watchObservedRunningTime="2026-07-22 02:26:13.240701898 +0000 UTC m=+404.149566892" Jul 22 02:26:15.489719 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:15.489684 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:26:15.489986 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:15.489751 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:26:16.206758 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:16.206723 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" event={"ID":"d1cd3d62-ed36-4952-ac62-89ca6313b20d","Type":"ContainerStarted","Data":"8d9267fb48fde79f08f14a8f29133dd312b0a37c4c29a57d367feaa400341699"} Jul 22 02:26:16.207009 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:16.206978 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:16.208492 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:16.208461 2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-jmk2j container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 02:26:16.208606 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:16.208529 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" podUID="d1cd3d62-ed36-4952-ac62-89ca6313b20d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 02:26:16.950729 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:16.950675 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" podStartSLOduration=3.044460812 podStartE2EDuration="8.950653299s" totalImagesPullingTime="5.906192487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:26:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:26:16.307340233 +0000 UTC m=+407.216205232" watchObservedRunningTime="2026-07-22 02:26:16.950653299 +0000 UTC m=+407.859518306" Jul 22 02:26:16.952179 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:16.952158 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-57frf"] Jul 22 02:26:16.954913 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:16.954897 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-57frf" Jul 22 02:26:16.957806 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:16.957779 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 22 02:26:16.957949 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:16.957782 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4ft64\"" Jul 22 02:26:16.966127 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:16.966103 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-74cxr"] Jul 22 02:26:16.969498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:16.969480 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-74cxr" Jul 22 02:26:16.973313 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:16.973294 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 22 02:26:16.973422 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:16.973379 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-pnh4g\"" Jul 22 02:26:16.976498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:16.976474 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-57frf"] Jul 22 02:26:16.987253 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:16.987213 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jklq\" (UniqueName: \"kubernetes.io/projected/5cd5e93b-ab6c-464b-a914-02ee4d72f695-kube-api-access-9jklq\") pod \"kserve-controller-manager-d6f4b65b8-57frf\" (UID: \"5cd5e93b-ab6c-464b-a914-02ee4d72f695\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-57frf" Jul 22 02:26:16.987414 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:16.987297 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cd5e93b-ab6c-464b-a914-02ee4d72f695-cert\") pod \"kserve-controller-manager-d6f4b65b8-57frf\" (UID: \"5cd5e93b-ab6c-464b-a914-02ee4d72f695\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-57frf" Jul 22 02:26:16.997371 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:16.997247 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-74cxr"] Jul 22 02:26:17.088556 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.088511 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhz5w\" (UniqueName: \"kubernetes.io/projected/32de991c-21b2-4846-94b6-52923946c2d4-kube-api-access-zhz5w\") pod \"llmisvc-controller-manager-777c6bd7d5-74cxr\" (UID: \"32de991c-21b2-4846-94b6-52923946c2d4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-74cxr" Jul 22 02:26:17.088738 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.088567 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cd5e93b-ab6c-464b-a914-02ee4d72f695-cert\") pod \"kserve-controller-manager-d6f4b65b8-57frf\" (UID: \"5cd5e93b-ab6c-464b-a914-02ee4d72f695\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-57frf" Jul 22 02:26:17.088738 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.088620 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32de991c-21b2-4846-94b6-52923946c2d4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-74cxr\" (UID: \"32de991c-21b2-4846-94b6-52923946c2d4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-74cxr" Jul 22 02:26:17.088738 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:26:17.088681 2557 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 22 02:26:17.088738 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.088719 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jklq\" (UniqueName: \"kubernetes.io/projected/5cd5e93b-ab6c-464b-a914-02ee4d72f695-kube-api-access-9jklq\") pod \"kserve-controller-manager-d6f4b65b8-57frf\" (UID: \"5cd5e93b-ab6c-464b-a914-02ee4d72f695\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-57frf" Jul 22 02:26:17.088738 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:26:17.088738 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cd5e93b-ab6c-464b-a914-02ee4d72f695-cert podName:5cd5e93b-ab6c-464b-a914-02ee4d72f695 nodeName:}" failed. No retries permitted until 2026-07-22 02:26:17.588716614 +0000 UTC m=+408.497581591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5cd5e93b-ab6c-464b-a914-02ee4d72f695-cert") pod "kserve-controller-manager-d6f4b65b8-57frf" (UID: "5cd5e93b-ab6c-464b-a914-02ee4d72f695") : secret "kserve-webhook-server-cert" not found Jul 22 02:26:17.111643 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.111618 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jklq\" (UniqueName: \"kubernetes.io/projected/5cd5e93b-ab6c-464b-a914-02ee4d72f695-kube-api-access-9jklq\") pod \"kserve-controller-manager-d6f4b65b8-57frf\" (UID: \"5cd5e93b-ab6c-464b-a914-02ee4d72f695\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-57frf" Jul 22 02:26:17.189587 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.189552 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhz5w\" (UniqueName: \"kubernetes.io/projected/32de991c-21b2-4846-94b6-52923946c2d4-kube-api-access-zhz5w\") pod \"llmisvc-controller-manager-777c6bd7d5-74cxr\" (UID: \"32de991c-21b2-4846-94b6-52923946c2d4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-74cxr" Jul 22 02:26:17.189736 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.189612 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32de991c-21b2-4846-94b6-52923946c2d4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-74cxr\" (UID: \"32de991c-21b2-4846-94b6-52923946c2d4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-74cxr" Jul 22 02:26:17.189736 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:26:17.189727 2557 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 22 02:26:17.189844 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:26:17.189788 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32de991c-21b2-4846-94b6-52923946c2d4-cert podName:32de991c-21b2-4846-94b6-52923946c2d4 nodeName:}" failed. No retries permitted until 2026-07-22 02:26:17.68977038 +0000 UTC m=+408.598635361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/32de991c-21b2-4846-94b6-52923946c2d4-cert") pod "llmisvc-controller-manager-777c6bd7d5-74cxr" (UID: "32de991c-21b2-4846-94b6-52923946c2d4") : secret "llmisvc-webhook-server-cert" not found Jul 22 02:26:17.212510 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.212470 2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-jmk2j container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 02:26:17.212642 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.212540 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" podUID="d1cd3d62-ed36-4952-ac62-89ca6313b20d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 02:26:17.228281 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.228257 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhz5w\" (UniqueName: \"kubernetes.io/projected/32de991c-21b2-4846-94b6-52923946c2d4-kube-api-access-zhz5w\") pod \"llmisvc-controller-manager-777c6bd7d5-74cxr\" (UID: \"32de991c-21b2-4846-94b6-52923946c2d4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-74cxr" Jul 22 02:26:17.593750 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.593660 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cd5e93b-ab6c-464b-a914-02ee4d72f695-cert\") pod \"kserve-controller-manager-d6f4b65b8-57frf\" (UID: \"5cd5e93b-ab6c-464b-a914-02ee4d72f695\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-57frf" Jul 22 02:26:17.596256 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.596228 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cd5e93b-ab6c-464b-a914-02ee4d72f695-cert\") pod \"kserve-controller-manager-d6f4b65b8-57frf\" (UID: \"5cd5e93b-ab6c-464b-a914-02ee4d72f695\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-57frf" Jul 22 02:26:17.694750 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.694714 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32de991c-21b2-4846-94b6-52923946c2d4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-74cxr\" (UID: \"32de991c-21b2-4846-94b6-52923946c2d4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-74cxr" Jul 22 02:26:17.697465 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.697443 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32de991c-21b2-4846-94b6-52923946c2d4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-74cxr\" (UID: \"32de991c-21b2-4846-94b6-52923946c2d4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-74cxr" Jul 22 02:26:17.866939 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.866853 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-57frf" Jul 22 02:26:17.881338 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.881313 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-74cxr" Jul 22 02:26:17.930341 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.929457 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f"] Jul 22 02:26:17.934504 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.934479 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:17.947525 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.939328 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-9nfdt\"" Jul 22 02:26:17.951204 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.951171 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f"] Jul 22 02:26:17.998636 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.998262 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/218745d1-9832-4760-b46f-3661fa7f4e5a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:17.998636 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.998294 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/218745d1-9832-4760-b46f-3661fa7f4e5a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:17.998636 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.998316 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/218745d1-9832-4760-b46f-3661fa7f4e5a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:17.998636 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.998344 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/218745d1-9832-4760-b46f-3661fa7f4e5a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:17.998636 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.998363 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/218745d1-9832-4760-b46f-3661fa7f4e5a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:17.998636 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.998401 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbndr\" (UniqueName: \"kubernetes.io/projected/218745d1-9832-4760-b46f-3661fa7f4e5a-kube-api-access-lbndr\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:17.998636 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.998434 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/218745d1-9832-4760-b46f-3661fa7f4e5a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:17.998636 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.998464 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/218745d1-9832-4760-b46f-3661fa7f4e5a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:17.998636 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:17.998495 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/218745d1-9832-4760-b46f-3661fa7f4e5a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.035907 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.035885 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-57frf"] Jul 22 02:26:18.038287 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:26:18.038258 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cd5e93b_ab6c_464b_a914_02ee4d72f695.slice/crio-020ad40a60e49e8d3018ae7c18c466dd9b040a70a1a18dcaa0c5b4c0c5f71213 WatchSource:0}: Error finding container 020ad40a60e49e8d3018ae7c18c466dd9b040a70a1a18dcaa0c5b4c0c5f71213: Status 404 returned error can't find the container with id 020ad40a60e49e8d3018ae7c18c466dd9b040a70a1a18dcaa0c5b4c0c5f71213 Jul 22 02:26:18.057289 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.057262 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-74cxr"] Jul 22 02:26:18.059678 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:26:18.059650 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod32de991c_21b2_4846_94b6_52923946c2d4.slice/crio-e121fb5572f9a735f9c7c6f413325137e1b0a7e6aca08f7a8a4011a41a69ee54 WatchSource:0}: Error finding container e121fb5572f9a735f9c7c6f413325137e1b0a7e6aca08f7a8a4011a41a69ee54: Status 404 returned error can't find the container with id e121fb5572f9a735f9c7c6f413325137e1b0a7e6aca08f7a8a4011a41a69ee54 Jul 22 02:26:18.099460 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.099433 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/218745d1-9832-4760-b46f-3661fa7f4e5a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.099636 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.099470 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/218745d1-9832-4760-b46f-3661fa7f4e5a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.099636 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.099504 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/218745d1-9832-4760-b46f-3661fa7f4e5a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.099636 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.099547 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/218745d1-9832-4760-b46f-3661fa7f4e5a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.099636 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.099574 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/218745d1-9832-4760-b46f-3661fa7f4e5a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.099636 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.099613 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbndr\" (UniqueName: \"kubernetes.io/projected/218745d1-9832-4760-b46f-3661fa7f4e5a-kube-api-access-lbndr\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.099901 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.099646 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/218745d1-9832-4760-b46f-3661fa7f4e5a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.099901 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.099679 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/218745d1-9832-4760-b46f-3661fa7f4e5a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.099901 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.099713 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/218745d1-9832-4760-b46f-3661fa7f4e5a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.100048 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.099925 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/218745d1-9832-4760-b46f-3661fa7f4e5a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.100182 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.100157 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/218745d1-9832-4760-b46f-3661fa7f4e5a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.100316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.100294 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/218745d1-9832-4760-b46f-3661fa7f4e5a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.100377 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.100335 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/218745d1-9832-4760-b46f-3661fa7f4e5a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.100433 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.100388 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/218745d1-9832-4760-b46f-3661fa7f4e5a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.102333 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.102314 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/218745d1-9832-4760-b46f-3661fa7f4e5a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.102426 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.102407 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/218745d1-9832-4760-b46f-3661fa7f4e5a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.109961 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.109940 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbndr\" (UniqueName: \"kubernetes.io/projected/218745d1-9832-4760-b46f-3661fa7f4e5a-kube-api-access-lbndr\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.110228 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.110213 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/218745d1-9832-4760-b46f-3661fa7f4e5a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nnp5f\" (UID: \"218745d1-9832-4760-b46f-3661fa7f4e5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.216964 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.216924 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-74cxr" event={"ID":"32de991c-21b2-4846-94b6-52923946c2d4","Type":"ContainerStarted","Data":"e121fb5572f9a735f9c7c6f413325137e1b0a7e6aca08f7a8a4011a41a69ee54"} Jul 22 02:26:18.218113 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.218087 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-57frf" event={"ID":"5cd5e93b-ab6c-464b-a914-02ee4d72f695","Type":"ContainerStarted","Data":"020ad40a60e49e8d3018ae7c18c466dd9b040a70a1a18dcaa0c5b4c0c5f71213"} Jul 22 02:26:18.251344 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.251317 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:18.405408 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.405386 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f"] Jul 22 02:26:18.407896 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:26:18.407867 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod218745d1_9832_4760_b46f_3661fa7f4e5a.slice/crio-caa5b93cf6212f5ff261849e4c7c91e7c48b590dc00835adedb07a9a0da7702f WatchSource:0}: Error finding container caa5b93cf6212f5ff261849e4c7c91e7c48b590dc00835adedb07a9a0da7702f: Status 404 returned error can't find the container with id caa5b93cf6212f5ff261849e4c7c91e7c48b590dc00835adedb07a9a0da7702f Jul 22 02:26:18.773133 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.773097 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j"] Jul 22 02:26:18.773351 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.773321 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" podUID="d1cd3d62-ed36-4952-ac62-89ca6313b20d" containerName="discovery" containerID="cri-o://8d9267fb48fde79f08f14a8f29133dd312b0a37c4c29a57d367feaa400341699" gracePeriod=30 Jul 22 02:26:18.774187 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:18.774167 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:19.225778 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.225744 2557 generic.go:356] "Generic (PLEG): container finished" podID="d1cd3d62-ed36-4952-ac62-89ca6313b20d" containerID="8d9267fb48fde79f08f14a8f29133dd312b0a37c4c29a57d367feaa400341699" exitCode=0 Jul 22 02:26:19.226239 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.225846 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" event={"ID":"d1cd3d62-ed36-4952-ac62-89ca6313b20d","Type":"ContainerDied","Data":"8d9267fb48fde79f08f14a8f29133dd312b0a37c4c29a57d367feaa400341699"} Jul 22 02:26:19.227264 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.227239 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" event={"ID":"218745d1-9832-4760-b46f-3661fa7f4e5a","Type":"ContainerStarted","Data":"caa5b93cf6212f5ff261849e4c7c91e7c48b590dc00835adedb07a9a0da7702f"} Jul 22 02:26:19.276712 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.276689 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:19.311560 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.311525 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-kubeconfig\") pod \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " Jul 22 02:26:19.311728 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.311578 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-token\") pod \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " Jul 22 02:26:19.311728 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.311625 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-csr-ca-configmap\") pod \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " Jul 22 02:26:19.311728 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.311664 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-cacerts\") pod \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " Jul 22 02:26:19.311728 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.311704 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-csr-dns-cert\") pod \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " Jul 22 02:26:19.311977 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.311738 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vngxq\" (UniqueName: \"kubernetes.io/projected/d1cd3d62-ed36-4952-ac62-89ca6313b20d-kube-api-access-vngxq\") pod \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " Jul 22 02:26:19.311977 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.311787 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1cd3d62-ed36-4952-ac62-89ca6313b20d-local-certs\") pod \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\" (UID: \"d1cd3d62-ed36-4952-ac62-89ca6313b20d\") " Jul 22 02:26:19.312075 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.312034 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "d1cd3d62-ed36-4952-ac62-89ca6313b20d" (UID: "d1cd3d62-ed36-4952-ac62-89ca6313b20d"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:26:19.312469 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.312151 2557 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-csr-ca-configmap\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:26:19.316246 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.316205 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "d1cd3d62-ed36-4952-ac62-89ca6313b20d" (UID: "d1cd3d62-ed36-4952-ac62-89ca6313b20d"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:26:19.316246 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.316211 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1cd3d62-ed36-4952-ac62-89ca6313b20d-kube-api-access-vngxq" (OuterVolumeSpecName: "kube-api-access-vngxq") pod "d1cd3d62-ed36-4952-ac62-89ca6313b20d" (UID: "d1cd3d62-ed36-4952-ac62-89ca6313b20d"). InnerVolumeSpecName "kube-api-access-vngxq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:26:19.316417 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.316288 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1cd3d62-ed36-4952-ac62-89ca6313b20d-local-certs" (OuterVolumeSpecName: "local-certs") pod "d1cd3d62-ed36-4952-ac62-89ca6313b20d" (UID: "d1cd3d62-ed36-4952-ac62-89ca6313b20d"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:26:19.316417 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.316293 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-cacerts" (OuterVolumeSpecName: "cacerts") pod "d1cd3d62-ed36-4952-ac62-89ca6313b20d" (UID: "d1cd3d62-ed36-4952-ac62-89ca6313b20d"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:26:19.316801 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.316761 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "d1cd3d62-ed36-4952-ac62-89ca6313b20d" (UID: "d1cd3d62-ed36-4952-ac62-89ca6313b20d"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:26:19.316918 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.316871 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-token" (OuterVolumeSpecName: "istio-token") pod "d1cd3d62-ed36-4952-ac62-89ca6313b20d" (UID: "d1cd3d62-ed36-4952-ac62-89ca6313b20d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:26:19.413139 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.413051 2557 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-cacerts\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:26:19.413139 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.413093 2557 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-csr-dns-cert\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:26:19.413139 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.413110 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vngxq\" (UniqueName: \"kubernetes.io/projected/d1cd3d62-ed36-4952-ac62-89ca6313b20d-kube-api-access-vngxq\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:26:19.413139 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.413118 2557 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1cd3d62-ed36-4952-ac62-89ca6313b20d-local-certs\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:26:19.413139 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.413127 2557 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-kubeconfig\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:26:19.413139 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:19.413134 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1cd3d62-ed36-4952-ac62-89ca6313b20d-istio-token\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:26:20.233312 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:20.233283 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" Jul 22 02:26:20.233721 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:20.233318 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j" event={"ID":"d1cd3d62-ed36-4952-ac62-89ca6313b20d","Type":"ContainerDied","Data":"321d0dfdfc7ee71030e721136f6804afeeba099d2aeb4c35a210b4104487abb0"} Jul 22 02:26:20.233721 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:20.233379 2557 scope.go:117] "RemoveContainer" containerID="8d9267fb48fde79f08f14a8f29133dd312b0a37c4c29a57d367feaa400341699" Jul 22 02:26:20.253205 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:20.253166 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j"] Jul 22 02:26:20.260543 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:20.260504 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jmk2j"] Jul 22 02:26:21.239243 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:21.239152 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-57frf" event={"ID":"5cd5e93b-ab6c-464b-a914-02ee4d72f695","Type":"ContainerStarted","Data":"85588dc2853f4c6e325e6708ef27763189a2e3c155269a745c2b40f135887565"} Jul 22 02:26:21.239698 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:21.239261 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-57frf" Jul 22 02:26:21.241598 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:21.241559 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-74cxr" event={"ID":"32de991c-21b2-4846-94b6-52923946c2d4","Type":"ContainerStarted","Data":"803f8ee602603abe66360b272d4128834759ad55393f05ae7aca8d57d17715f5"} Jul 22 02:26:21.241727 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:21.241712 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-74cxr" Jul 22 02:26:21.280269 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:21.280221 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-57frf" podStartSLOduration=2.520713809 podStartE2EDuration="5.280198851s" totalImagesPullingTime="2.759485042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:26:21.277913226 +0000 UTC m=+412.186778226" watchObservedRunningTime="2026-07-22 02:26:21.280198851 +0000 UTC m=+412.189063848" Jul 22 02:26:21.308428 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:21.308378 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-74cxr" podStartSLOduration=2.565119875 podStartE2EDuration="5.308364371s" totalImagesPullingTime="2.743244496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:26:21.306525577 +0000 UTC m=+412.215390572" watchObservedRunningTime="2026-07-22 02:26:21.308364371 +0000 UTC m=+412.217229365" Jul 22 02:26:21.609211 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:21.609142 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1cd3d62-ed36-4952-ac62-89ca6313b20d" path="/var/lib/kubelet/pods/d1cd3d62-ed36-4952-ac62-89ca6313b20d/volumes" Jul 22 02:26:24.198487 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:24.198449 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-wfh7b" Jul 22 02:26:24.200762 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:24.200734 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-bqxlc" Jul 22 02:26:24.586074 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:24.586039 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:26:24.586216 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:24.586106 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:26:24.586216 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:24.586136 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:26:25.259313 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:25.259277 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" event={"ID":"218745d1-9832-4760-b46f-3661fa7f4e5a","Type":"ContainerStarted","Data":"8f2df8d1b238cea10c4bef31104ae5a4cc808089d86ebc261a032b14c39712f6"} Jul 22 02:26:25.284482 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:25.284424 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" podStartSLOduration=2.10903399 podStartE2EDuration="8.284405958s" totalImagesPullingTime="6.175371968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:26:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:26:25.282242625 +0000 UTC m=+416.191107620" watchObservedRunningTime="2026-07-22 02:26:25.284405958 +0000 UTC m=+416.193270954" Jul 22 02:26:26.251525 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:26.251491 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:26.255847 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:26.255800 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:26.262874 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:26.262851 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:26.263534 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:26.263520 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nnp5f" Jul 22 02:26:52.248289 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:52.248202 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-74cxr" Jul 22 02:26:52.251054 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:26:52.251031 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-57frf" Jul 22 02:27:06.635397 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:06.635361 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rhbb5"] Jul 22 02:27:06.635756 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:06.635648 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1cd3d62-ed36-4952-ac62-89ca6313b20d" containerName="discovery" Jul 22 02:27:06.635756 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:06.635658 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1cd3d62-ed36-4952-ac62-89ca6313b20d" containerName="discovery" Jul 22 02:27:06.635756 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:06.635708 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1cd3d62-ed36-4952-ac62-89ca6313b20d" containerName="discovery" Jul 22 02:27:06.650379 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:06.650342 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rhbb5" Jul 22 02:27:06.654752 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:06.654731 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 02:27:06.655961 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:06.655940 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 02:27:06.655961 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:06.655953 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-f6ct2\"" Jul 22 02:27:06.664096 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:06.664075 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rhbb5"] Jul 22 02:27:06.703442 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:06.703412 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4l7r\" (UniqueName: \"kubernetes.io/projected/809652bf-a77f-4a1d-b51b-3e9c78ae03d7-kube-api-access-w4l7r\") pod \"kuadrant-operator-catalog-rhbb5\" (UID: \"809652bf-a77f-4a1d-b51b-3e9c78ae03d7\") " pod="kuadrant-system/kuadrant-operator-catalog-rhbb5" Jul 22 02:27:06.804750 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:06.804720 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4l7r\" (UniqueName: \"kubernetes.io/projected/809652bf-a77f-4a1d-b51b-3e9c78ae03d7-kube-api-access-w4l7r\") pod \"kuadrant-operator-catalog-rhbb5\" (UID: \"809652bf-a77f-4a1d-b51b-3e9c78ae03d7\") " pod="kuadrant-system/kuadrant-operator-catalog-rhbb5" Jul 22 02:27:06.824137 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:06.824107 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4l7r\" (UniqueName: \"kubernetes.io/projected/809652bf-a77f-4a1d-b51b-3e9c78ae03d7-kube-api-access-w4l7r\") pod \"kuadrant-operator-catalog-rhbb5\" (UID: \"809652bf-a77f-4a1d-b51b-3e9c78ae03d7\") " pod="kuadrant-system/kuadrant-operator-catalog-rhbb5" Jul 22 02:27:06.960269 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:06.960189 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rhbb5" Jul 22 02:27:07.111608 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:07.111585 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rhbb5"] Jul 22 02:27:07.114367 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:27:07.114332 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod809652bf_a77f_4a1d_b51b_3e9c78ae03d7.slice/crio-5bfcc4a5b8d72b5269e9ccd9a0a8d474686b1addc82189bf585910bcc0bbec14 WatchSource:0}: Error finding container 5bfcc4a5b8d72b5269e9ccd9a0a8d474686b1addc82189bf585910bcc0bbec14: Status 404 returned error can't find the container with id 5bfcc4a5b8d72b5269e9ccd9a0a8d474686b1addc82189bf585910bcc0bbec14 Jul 22 02:27:07.171609 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:07.171576 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rhbb5"] Jul 22 02:27:07.214529 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:07.214505 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-npftk"] Jul 22 02:27:07.219063 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:07.219048 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-npftk" Jul 22 02:27:07.231991 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:07.231971 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-npftk"] Jul 22 02:27:07.307556 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:07.307519 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6flr\" (UniqueName: \"kubernetes.io/projected/4cbddcc4-1c96-4012-a13d-b6859697723a-kube-api-access-t6flr\") pod \"kuadrant-operator-catalog-npftk\" (UID: \"4cbddcc4-1c96-4012-a13d-b6859697723a\") " pod="kuadrant-system/kuadrant-operator-catalog-npftk" Jul 22 02:27:07.408149 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:07.408106 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6flr\" (UniqueName: \"kubernetes.io/projected/4cbddcc4-1c96-4012-a13d-b6859697723a-kube-api-access-t6flr\") pod \"kuadrant-operator-catalog-npftk\" (UID: \"4cbddcc4-1c96-4012-a13d-b6859697723a\") " pod="kuadrant-system/kuadrant-operator-catalog-npftk" Jul 22 02:27:07.418688 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:07.418660 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6flr\" (UniqueName: \"kubernetes.io/projected/4cbddcc4-1c96-4012-a13d-b6859697723a-kube-api-access-t6flr\") pod \"kuadrant-operator-catalog-npftk\" (UID: \"4cbddcc4-1c96-4012-a13d-b6859697723a\") " pod="kuadrant-system/kuadrant-operator-catalog-npftk" Jul 22 02:27:07.428986 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:07.428959 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rhbb5" event={"ID":"809652bf-a77f-4a1d-b51b-3e9c78ae03d7","Type":"ContainerStarted","Data":"5bfcc4a5b8d72b5269e9ccd9a0a8d474686b1addc82189bf585910bcc0bbec14"} Jul 22 02:27:07.527882 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:07.527781 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-npftk" Jul 22 02:27:07.659035 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:07.659009 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-npftk"] Jul 22 02:27:07.661163 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:27:07.661135 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cbddcc4_1c96_4012_a13d_b6859697723a.slice/crio-e0a127d041124cf6acb1e3ba86aacf640f19e475def262b8a6ed054fcc47411d WatchSource:0}: Error finding container e0a127d041124cf6acb1e3ba86aacf640f19e475def262b8a6ed054fcc47411d: Status 404 returned error can't find the container with id e0a127d041124cf6acb1e3ba86aacf640f19e475def262b8a6ed054fcc47411d Jul 22 02:27:08.434985 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:08.434955 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-npftk" event={"ID":"4cbddcc4-1c96-4012-a13d-b6859697723a","Type":"ContainerStarted","Data":"e0a127d041124cf6acb1e3ba86aacf640f19e475def262b8a6ed054fcc47411d"} Jul 22 02:27:09.440673 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:09.440563 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rhbb5" event={"ID":"809652bf-a77f-4a1d-b51b-3e9c78ae03d7","Type":"ContainerStarted","Data":"c4e630bc18e299b05e2a852d9ca4cdf8cf12cafed2c808b1965d224d5763335f"} Jul 22 02:27:09.440673 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:09.440584 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-rhbb5" podUID="809652bf-a77f-4a1d-b51b-3e9c78ae03d7" containerName="registry-server" containerID="cri-o://c4e630bc18e299b05e2a852d9ca4cdf8cf12cafed2c808b1965d224d5763335f" gracePeriod=2 Jul 22 02:27:09.442349 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:09.442319 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-npftk" event={"ID":"4cbddcc4-1c96-4012-a13d-b6859697723a","Type":"ContainerStarted","Data":"95038acb3216d1aeebb1a81c824e4d1431e40176af691059a18e5218af4066b3"} Jul 22 02:27:09.481885 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:09.481848 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-rhbb5" podStartSLOduration=1.560998166 podStartE2EDuration="3.481812999s" totalImagesPullingTime="1.920814833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:27:09.481723169 +0000 UTC m=+460.390588163" watchObservedRunningTime="2026-07-22 02:27:09.481812999 +0000 UTC m=+460.390677995" Jul 22 02:27:09.513467 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:09.513426 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-npftk" podStartSLOduration=1.138924974 podStartE2EDuration="2.513410331s" totalImagesPullingTime="1.374485357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:27:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:27:09.511725384 +0000 UTC m=+460.420590392" watchObservedRunningTime="2026-07-22 02:27:09.513410331 +0000 UTC m=+460.422275329" Jul 22 02:27:09.687906 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:09.687883 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rhbb5" Jul 22 02:27:09.728463 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:09.728437 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4l7r\" (UniqueName: \"kubernetes.io/projected/809652bf-a77f-4a1d-b51b-3e9c78ae03d7-kube-api-access-w4l7r\") pod \"809652bf-a77f-4a1d-b51b-3e9c78ae03d7\" (UID: \"809652bf-a77f-4a1d-b51b-3e9c78ae03d7\") " Jul 22 02:27:09.730662 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:09.730635 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/809652bf-a77f-4a1d-b51b-3e9c78ae03d7-kube-api-access-w4l7r" (OuterVolumeSpecName: "kube-api-access-w4l7r") pod "809652bf-a77f-4a1d-b51b-3e9c78ae03d7" (UID: "809652bf-a77f-4a1d-b51b-3e9c78ae03d7"). InnerVolumeSpecName "kube-api-access-w4l7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:27:09.829444 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:09.829397 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4l7r\" (UniqueName: \"kubernetes.io/projected/809652bf-a77f-4a1d-b51b-3e9c78ae03d7-kube-api-access-w4l7r\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:27:10.447479 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:10.447392 2557 generic.go:356] "Generic (PLEG): container finished" podID="809652bf-a77f-4a1d-b51b-3e9c78ae03d7" containerID="c4e630bc18e299b05e2a852d9ca4cdf8cf12cafed2c808b1965d224d5763335f" exitCode=0 Jul 22 02:27:10.447479 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:10.447431 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rhbb5" Jul 22 02:27:10.447479 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:10.447461 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rhbb5" event={"ID":"809652bf-a77f-4a1d-b51b-3e9c78ae03d7","Type":"ContainerDied","Data":"c4e630bc18e299b05e2a852d9ca4cdf8cf12cafed2c808b1965d224d5763335f"} Jul 22 02:27:10.447989 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:10.447499 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rhbb5" event={"ID":"809652bf-a77f-4a1d-b51b-3e9c78ae03d7","Type":"ContainerDied","Data":"5bfcc4a5b8d72b5269e9ccd9a0a8d474686b1addc82189bf585910bcc0bbec14"} Jul 22 02:27:10.447989 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:10.447517 2557 scope.go:117] "RemoveContainer" containerID="c4e630bc18e299b05e2a852d9ca4cdf8cf12cafed2c808b1965d224d5763335f" Jul 22 02:27:10.456750 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:10.456735 2557 scope.go:117] "RemoveContainer" containerID="c4e630bc18e299b05e2a852d9ca4cdf8cf12cafed2c808b1965d224d5763335f" Jul 22 02:27:10.457044 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:27:10.457020 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4e630bc18e299b05e2a852d9ca4cdf8cf12cafed2c808b1965d224d5763335f\": container with ID starting with c4e630bc18e299b05e2a852d9ca4cdf8cf12cafed2c808b1965d224d5763335f not found: ID does not exist" containerID="c4e630bc18e299b05e2a852d9ca4cdf8cf12cafed2c808b1965d224d5763335f" Jul 22 02:27:10.457093 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:10.457053 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4e630bc18e299b05e2a852d9ca4cdf8cf12cafed2c808b1965d224d5763335f"} err="failed to get container status \"c4e630bc18e299b05e2a852d9ca4cdf8cf12cafed2c808b1965d224d5763335f\": rpc error: code = NotFound desc = could not find container \"c4e630bc18e299b05e2a852d9ca4cdf8cf12cafed2c808b1965d224d5763335f\": container with ID starting with c4e630bc18e299b05e2a852d9ca4cdf8cf12cafed2c808b1965d224d5763335f not found: ID does not exist" Jul 22 02:27:10.470198 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:10.470179 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rhbb5"] Jul 22 02:27:10.478210 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:10.475925 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rhbb5"] Jul 22 02:27:11.608800 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:11.608765 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="809652bf-a77f-4a1d-b51b-3e9c78ae03d7" path="/var/lib/kubelet/pods/809652bf-a77f-4a1d-b51b-3e9c78ae03d7/volumes" Jul 22 02:27:17.528804 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:17.528756 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-npftk" Jul 22 02:27:17.529327 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:17.528862 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-npftk" Jul 22 02:27:17.551016 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:17.550992 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-npftk" Jul 22 02:27:18.502254 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:18.502228 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-npftk" Jul 22 02:27:37.924521 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:37.924482 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7cl6l"] Jul 22 02:27:37.925063 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:37.925005 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="809652bf-a77f-4a1d-b51b-3e9c78ae03d7" containerName="registry-server" Jul 22 02:27:37.925063 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:37.925025 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="809652bf-a77f-4a1d-b51b-3e9c78ae03d7" containerName="registry-server" Jul 22 02:27:37.925191 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:37.925113 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="809652bf-a77f-4a1d-b51b-3e9c78ae03d7" containerName="registry-server" Jul 22 02:27:37.930083 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:37.930061 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7cl6l" Jul 22 02:27:37.934453 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:37.934437 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-7454g\"" Jul 22 02:27:37.934544 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:37.934457 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 22 02:27:37.944215 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:37.944193 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7cl6l"] Jul 22 02:27:38.055323 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:38.055282 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xvzf\" (UniqueName: \"kubernetes.io/projected/beac8017-6dbc-4668-83a9-47134c978970-kube-api-access-8xvzf\") pod \"dns-operator-controller-manager-864dd78c4c-7cl6l\" (UID: \"beac8017-6dbc-4668-83a9-47134c978970\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7cl6l" Jul 22 02:27:38.156351 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:38.156309 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xvzf\" (UniqueName: \"kubernetes.io/projected/beac8017-6dbc-4668-83a9-47134c978970-kube-api-access-8xvzf\") pod \"dns-operator-controller-manager-864dd78c4c-7cl6l\" (UID: \"beac8017-6dbc-4668-83a9-47134c978970\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7cl6l" Jul 22 02:27:38.166290 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:38.166265 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xvzf\" (UniqueName: \"kubernetes.io/projected/beac8017-6dbc-4668-83a9-47134c978970-kube-api-access-8xvzf\") pod \"dns-operator-controller-manager-864dd78c4c-7cl6l\" (UID: \"beac8017-6dbc-4668-83a9-47134c978970\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7cl6l" Jul 22 02:27:38.240022 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:38.239997 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7cl6l" Jul 22 02:27:38.368798 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:38.368768 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7cl6l"] Jul 22 02:27:38.372310 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:27:38.372285 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeac8017_6dbc_4668_83a9_47134c978970.slice/crio-8b1c59aa2e525df854ce726f262884af707eb19c410b869beb3cd068f037e6ba WatchSource:0}: Error finding container 8b1c59aa2e525df854ce726f262884af707eb19c410b869beb3cd068f037e6ba: Status 404 returned error can't find the container with id 8b1c59aa2e525df854ce726f262884af707eb19c410b869beb3cd068f037e6ba Jul 22 02:27:38.565275 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:38.565194 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7cl6l" event={"ID":"beac8017-6dbc-4668-83a9-47134c978970","Type":"ContainerStarted","Data":"8b1c59aa2e525df854ce726f262884af707eb19c410b869beb3cd068f037e6ba"} Jul 22 02:27:40.581055 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:40.580962 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7cl6l" event={"ID":"beac8017-6dbc-4668-83a9-47134c978970","Type":"ContainerStarted","Data":"60566e31226bbfe1b5a436b6f48bdc60524845f315fddc1de09eff6cf85c858f"} Jul 22 02:27:40.581055 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:40.581034 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7cl6l" Jul 22 02:27:40.610161 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:40.610112 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7cl6l" podStartSLOduration=1.689855441 podStartE2EDuration="3.610096595s" totalImagesPullingTime="1.920241154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:27:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:27:40.608672463 +0000 UTC m=+491.517537457" watchObservedRunningTime="2026-07-22 02:27:40.610096595 +0000 UTC m=+491.518961589" Jul 22 02:27:43.287153 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:43.287117 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7"] Jul 22 02:27:43.292437 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:43.292413 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" Jul 22 02:27:43.296569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:43.296549 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6vhqt\"" Jul 22 02:27:43.345881 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:43.345850 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7"] Jul 22 02:27:43.397401 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:43.397374 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch6rp\" (UniqueName: \"kubernetes.io/projected/9813117d-89f3-4796-848b-36b2b51fbdb4-kube-api-access-ch6rp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qtxm7\" (UID: \"9813117d-89f3-4796-848b-36b2b51fbdb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" Jul 22 02:27:43.397518 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:43.397424 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9813117d-89f3-4796-848b-36b2b51fbdb4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qtxm7\" (UID: \"9813117d-89f3-4796-848b-36b2b51fbdb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" Jul 22 02:27:43.498774 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:43.498742 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9813117d-89f3-4796-848b-36b2b51fbdb4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qtxm7\" (UID: \"9813117d-89f3-4796-848b-36b2b51fbdb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" Jul 22 02:27:43.498996 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:43.498807 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch6rp\" (UniqueName: \"kubernetes.io/projected/9813117d-89f3-4796-848b-36b2b51fbdb4-kube-api-access-ch6rp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qtxm7\" (UID: \"9813117d-89f3-4796-848b-36b2b51fbdb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" Jul 22 02:27:43.499245 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:43.499220 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9813117d-89f3-4796-848b-36b2b51fbdb4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qtxm7\" (UID: \"9813117d-89f3-4796-848b-36b2b51fbdb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" Jul 22 02:27:43.515289 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:43.515259 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch6rp\" (UniqueName: \"kubernetes.io/projected/9813117d-89f3-4796-848b-36b2b51fbdb4-kube-api-access-ch6rp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qtxm7\" (UID: \"9813117d-89f3-4796-848b-36b2b51fbdb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" Jul 22 02:27:43.602092 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:43.602012 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" Jul 22 02:27:43.758589 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:43.758558 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7"] Jul 22 02:27:43.759691 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:27:43.759665 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9813117d_89f3_4796_848b_36b2b51fbdb4.slice/crio-092b60509e11d14f6614ad9ccc4b79b8f29a4a72461bb8bb8c85322473e6612b WatchSource:0}: Error finding container 092b60509e11d14f6614ad9ccc4b79b8f29a4a72461bb8bb8c85322473e6612b: Status 404 returned error can't find the container with id 092b60509e11d14f6614ad9ccc4b79b8f29a4a72461bb8bb8c85322473e6612b Jul 22 02:27:44.600601 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:44.600565 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" event={"ID":"9813117d-89f3-4796-848b-36b2b51fbdb4","Type":"ContainerStarted","Data":"092b60509e11d14f6614ad9ccc4b79b8f29a4a72461bb8bb8c85322473e6612b"} Jul 22 02:27:47.818721 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:47.818682 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jxqwm"] Jul 22 02:27:47.822569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:47.822547 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jxqwm" Jul 22 02:27:47.829414 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:47.829387 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-q6pwr\"" Jul 22 02:27:47.860180 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:47.860147 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jxqwm"] Jul 22 02:27:47.936235 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:47.936202 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5szg\" (UniqueName: \"kubernetes.io/projected/b888f23c-adc1-46d7-91fb-53c1a676e036-kube-api-access-q5szg\") pod \"authorino-operator-75cc8d5cb7-jxqwm\" (UID: \"b888f23c-adc1-46d7-91fb-53c1a676e036\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jxqwm" Jul 22 02:27:48.037574 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:48.037534 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5szg\" (UniqueName: \"kubernetes.io/projected/b888f23c-adc1-46d7-91fb-53c1a676e036-kube-api-access-q5szg\") pod \"authorino-operator-75cc8d5cb7-jxqwm\" (UID: \"b888f23c-adc1-46d7-91fb-53c1a676e036\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jxqwm" Jul 22 02:27:48.085724 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:48.085648 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5szg\" (UniqueName: \"kubernetes.io/projected/b888f23c-adc1-46d7-91fb-53c1a676e036-kube-api-access-q5szg\") pod \"authorino-operator-75cc8d5cb7-jxqwm\" (UID: \"b888f23c-adc1-46d7-91fb-53c1a676e036\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jxqwm" Jul 22 02:27:48.134425 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:48.134396 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jxqwm" Jul 22 02:27:48.739382 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:48.739360 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jxqwm"] Jul 22 02:27:48.740911 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:27:48.740880 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb888f23c_adc1_46d7_91fb_53c1a676e036.slice/crio-118fb4496af356fa648f77dad6f5bae2671bbda752a9eec0e7c94e4a334a61cd WatchSource:0}: Error finding container 118fb4496af356fa648f77dad6f5bae2671bbda752a9eec0e7c94e4a334a61cd: Status 404 returned error can't find the container with id 118fb4496af356fa648f77dad6f5bae2671bbda752a9eec0e7c94e4a334a61cd Jul 22 02:27:49.628312 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:49.628271 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" event={"ID":"9813117d-89f3-4796-848b-36b2b51fbdb4","Type":"ContainerStarted","Data":"24fb1a8c7653c0ff6e61a986868933ab78e7c508eacce498e510c8a4d944e3dd"} Jul 22 02:27:49.628792 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:49.628345 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" Jul 22 02:27:49.629805 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:49.629782 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jxqwm" event={"ID":"b888f23c-adc1-46d7-91fb-53c1a676e036","Type":"ContainerStarted","Data":"118fb4496af356fa648f77dad6f5bae2671bbda752a9eec0e7c94e4a334a61cd"} Jul 22 02:27:49.656626 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:49.656591 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn"] Jul 22 02:27:49.660425 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:49.660400 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn" Jul 22 02:27:49.663563 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:49.663535 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-vdrcx\"" Jul 22 02:27:49.667862 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:49.667757 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" podStartSLOduration=1.783629644 podStartE2EDuration="6.667742594s" totalImagesPullingTime="4.88411295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:27:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:27:49.666670838 +0000 UTC m=+500.575535832" watchObservedRunningTime="2026-07-22 02:27:49.667742594 +0000 UTC m=+500.576607590" Jul 22 02:27:49.690240 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:49.690213 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn"] Jul 22 02:27:49.753238 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:49.753197 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpvvk\" (UniqueName: \"kubernetes.io/projected/2782815f-75b6-4619-82e6-f46a3dc11013-kube-api-access-cpvvk\") pod \"limitador-operator-controller-manager-859d7b78bd-59trn\" (UID: \"2782815f-75b6-4619-82e6-f46a3dc11013\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn" Jul 22 02:27:49.854047 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:49.854010 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpvvk\" (UniqueName: \"kubernetes.io/projected/2782815f-75b6-4619-82e6-f46a3dc11013-kube-api-access-cpvvk\") pod \"limitador-operator-controller-manager-859d7b78bd-59trn\" (UID: \"2782815f-75b6-4619-82e6-f46a3dc11013\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn" Jul 22 02:27:49.867965 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:49.867932 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpvvk\" (UniqueName: \"kubernetes.io/projected/2782815f-75b6-4619-82e6-f46a3dc11013-kube-api-access-cpvvk\") pod \"limitador-operator-controller-manager-859d7b78bd-59trn\" (UID: \"2782815f-75b6-4619-82e6-f46a3dc11013\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn" Jul 22 02:27:49.975906 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:49.975870 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn" Jul 22 02:27:50.154628 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:50.154598 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn"] Jul 22 02:27:50.155985 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:27:50.155957 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2782815f_75b6_4619_82e6_f46a3dc11013.slice/crio-635391b2a679133fcdc94bf33cf51b7e5a203e724626739349749b173e669104 WatchSource:0}: Error finding container 635391b2a679133fcdc94bf33cf51b7e5a203e724626739349749b173e669104: Status 404 returned error can't find the container with id 635391b2a679133fcdc94bf33cf51b7e5a203e724626739349749b173e669104 Jul 22 02:27:50.636893 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:50.636852 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn" event={"ID":"2782815f-75b6-4619-82e6-f46a3dc11013","Type":"ContainerStarted","Data":"635391b2a679133fcdc94bf33cf51b7e5a203e724626739349749b173e669104"} Jul 22 02:27:51.588313 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:51.588232 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7cl6l" Jul 22 02:27:52.649108 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:52.649066 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jxqwm" event={"ID":"b888f23c-adc1-46d7-91fb-53c1a676e036","Type":"ContainerStarted","Data":"728ee181e742bd9807d3e55bbd7fe5a749e3fab7c65e905bfc31e9091120d92c"} Jul 22 02:27:52.649504 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:52.649210 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jxqwm" Jul 22 02:27:52.650501 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:52.650478 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn" event={"ID":"2782815f-75b6-4619-82e6-f46a3dc11013","Type":"ContainerStarted","Data":"f514e7ec5db468028218a943f12edc73ec765fb2233eb4cd795629c666093cc1"} Jul 22 02:27:52.650601 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:52.650552 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn" Jul 22 02:27:52.675503 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:52.675457 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jxqwm" podStartSLOduration=3.113419918 podStartE2EDuration="5.675441628s" totalImagesPullingTime="2.56202171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:27:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:27:52.674739998 +0000 UTC m=+503.583605006" watchObservedRunningTime="2026-07-22 02:27:52.675441628 +0000 UTC m=+503.584306625" Jul 22 02:27:52.704120 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:27:52.704063 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn" podStartSLOduration=1.888326645 podStartE2EDuration="3.704042967s" totalImagesPullingTime="1.815716322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:27:52.702073856 +0000 UTC m=+503.610938850" watchObservedRunningTime="2026-07-22 02:27:52.704042967 +0000 UTC m=+503.612907963" Jul 22 02:28:00.640210 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:00.640178 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" Jul 22 02:28:03.127143 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.127110 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7"] Jul 22 02:28:03.127569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.127309 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" podUID="9813117d-89f3-4796-848b-36b2b51fbdb4" containerName="manager" containerID="cri-o://24fb1a8c7653c0ff6e61a986868933ab78e7c508eacce498e510c8a4d944e3dd" gracePeriod=2 Jul 22 02:28:03.139037 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.139007 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7"] Jul 22 02:28:03.171579 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.171552 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn"] Jul 22 02:28:03.171762 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.171735 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn" podUID="2782815f-75b6-4619-82e6-f46a3dc11013" containerName="manager" containerID="cri-o://f514e7ec5db468028218a943f12edc73ec765fb2233eb4cd795629c666093cc1" gracePeriod=2 Jul 22 02:28:03.173555 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.173533 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn" Jul 22 02:28:03.206168 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.206132 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn"] Jul 22 02:28:03.207385 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:28:03.207350 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qtxm7\" is forbidden: User \"system:node:ip-10-0-132-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-119.ec2.internal' and this object" podUID="9813117d-89f3-4796-848b-36b2b51fbdb4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" Jul 22 02:28:03.207593 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.207577 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j"] Jul 22 02:28:03.207893 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.207881 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9813117d-89f3-4796-848b-36b2b51fbdb4" containerName="manager" Jul 22 02:28:03.207940 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.207895 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9813117d-89f3-4796-848b-36b2b51fbdb4" containerName="manager" Jul 22 02:28:03.207940 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.207908 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2782815f-75b6-4619-82e6-f46a3dc11013" containerName="manager" Jul 22 02:28:03.207940 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.207914 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2782815f-75b6-4619-82e6-f46a3dc11013" containerName="manager" Jul 22 02:28:03.208026 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.207971 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9813117d-89f3-4796-848b-36b2b51fbdb4" containerName="manager" Jul 22 02:28:03.208026 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.207980 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2782815f-75b6-4619-82e6-f46a3dc11013" containerName="manager" Jul 22 02:28:03.210680 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.210664 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" Jul 22 02:28:03.224795 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:28:03.224706 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qtxm7\" is forbidden: User \"system:node:ip-10-0-132-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-119.ec2.internal' and this object" podUID="9813117d-89f3-4796-848b-36b2b51fbdb4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" Jul 22 02:28:03.224977 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.224871 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fwz5b"] Jul 22 02:28:03.228260 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.228231 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j"] Jul 22 02:28:03.228376 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.228329 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fwz5b" Jul 22 02:28:03.238726 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:28:03.238688 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-59trn\" is forbidden: User \"system:node:ip-10-0-132-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-119.ec2.internal' and this object" podUID="2782815f-75b6-4619-82e6-f46a3dc11013" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn" Jul 22 02:28:03.245895 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:28:03.245859 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-59trn\" is forbidden: User \"system:node:ip-10-0-132-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-119.ec2.internal' and this object" podUID="2782815f-75b6-4619-82e6-f46a3dc11013" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn" Jul 22 02:28:03.270658 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.270635 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg6jt\" (UniqueName: \"kubernetes.io/projected/9db3f385-a143-4024-be8a-9d10507eff8a-kube-api-access-kg6jt\") pod \"limitador-operator-controller-manager-859d7b78bd-fwz5b\" (UID: \"9db3f385-a143-4024-be8a-9d10507eff8a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fwz5b" Jul 22 02:28:03.270766 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.270710 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2602ef9a-1957-4cdd-9dd3-6891a77fa68f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vxr7j\" (UID: \"2602ef9a-1957-4cdd-9dd3-6891a77fa68f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" Jul 22 02:28:03.270766 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.270734 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkjg8\" (UniqueName: \"kubernetes.io/projected/2602ef9a-1957-4cdd-9dd3-6891a77fa68f-kube-api-access-wkjg8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vxr7j\" (UID: \"2602ef9a-1957-4cdd-9dd3-6891a77fa68f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" Jul 22 02:28:03.273151 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.273130 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fwz5b"] Jul 22 02:28:03.351501 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:28:03.351468 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qtxm7\" is forbidden: User \"system:node:ip-10-0-132-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-119.ec2.internal' and this object" podUID="9813117d-89f3-4796-848b-36b2b51fbdb4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" Jul 22 02:28:03.371115 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.371080 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2602ef9a-1957-4cdd-9dd3-6891a77fa68f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vxr7j\" (UID: \"2602ef9a-1957-4cdd-9dd3-6891a77fa68f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" Jul 22 02:28:03.371276 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.371128 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkjg8\" (UniqueName: \"kubernetes.io/projected/2602ef9a-1957-4cdd-9dd3-6891a77fa68f-kube-api-access-wkjg8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vxr7j\" (UID: \"2602ef9a-1957-4cdd-9dd3-6891a77fa68f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" Jul 22 02:28:03.371276 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.371169 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg6jt\" (UniqueName: \"kubernetes.io/projected/9db3f385-a143-4024-be8a-9d10507eff8a-kube-api-access-kg6jt\") pod \"limitador-operator-controller-manager-859d7b78bd-fwz5b\" (UID: \"9db3f385-a143-4024-be8a-9d10507eff8a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fwz5b" Jul 22 02:28:03.371542 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.371508 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2602ef9a-1957-4cdd-9dd3-6891a77fa68f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vxr7j\" (UID: \"2602ef9a-1957-4cdd-9dd3-6891a77fa68f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" Jul 22 02:28:03.387620 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.387557 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkjg8\" (UniqueName: \"kubernetes.io/projected/2602ef9a-1957-4cdd-9dd3-6891a77fa68f-kube-api-access-wkjg8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vxr7j\" (UID: \"2602ef9a-1957-4cdd-9dd3-6891a77fa68f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" Jul 22 02:28:03.395446 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.395421 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg6jt\" (UniqueName: \"kubernetes.io/projected/9db3f385-a143-4024-be8a-9d10507eff8a-kube-api-access-kg6jt\") pod \"limitador-operator-controller-manager-859d7b78bd-fwz5b\" (UID: \"9db3f385-a143-4024-be8a-9d10507eff8a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fwz5b" Jul 22 02:28:03.418470 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.418447 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" Jul 22 02:28:03.420994 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.420976 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn" Jul 22 02:28:03.421421 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:28:03.421401 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-59trn\" is forbidden: User \"system:node:ip-10-0-132-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-119.ec2.internal' and this object" podUID="2782815f-75b6-4619-82e6-f46a3dc11013" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn" Jul 22 02:28:03.423845 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:28:03.423799 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qtxm7\" is forbidden: User \"system:node:ip-10-0-132-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-119.ec2.internal' and this object" podUID="9813117d-89f3-4796-848b-36b2b51fbdb4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" Jul 22 02:28:03.426263 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:28:03.426241 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-59trn\" is forbidden: User \"system:node:ip-10-0-132-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-119.ec2.internal' and this object" podUID="2782815f-75b6-4619-82e6-f46a3dc11013" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn" Jul 22 02:28:03.439096 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:28:03.439076 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qtxm7\" is forbidden: User \"system:node:ip-10-0-132-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-119.ec2.internal' and this object" podUID="9813117d-89f3-4796-848b-36b2b51fbdb4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" Jul 22 02:28:03.471466 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.471436 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9813117d-89f3-4796-848b-36b2b51fbdb4-extensions-socket-volume\") pod \"9813117d-89f3-4796-848b-36b2b51fbdb4\" (UID: \"9813117d-89f3-4796-848b-36b2b51fbdb4\") " Jul 22 02:28:03.471612 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.471504 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ch6rp\" (UniqueName: \"kubernetes.io/projected/9813117d-89f3-4796-848b-36b2b51fbdb4-kube-api-access-ch6rp\") pod \"9813117d-89f3-4796-848b-36b2b51fbdb4\" (UID: \"9813117d-89f3-4796-848b-36b2b51fbdb4\") " Jul 22 02:28:03.471612 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.471545 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpvvk\" (UniqueName: \"kubernetes.io/projected/2782815f-75b6-4619-82e6-f46a3dc11013-kube-api-access-cpvvk\") pod \"2782815f-75b6-4619-82e6-f46a3dc11013\" (UID: \"2782815f-75b6-4619-82e6-f46a3dc11013\") " Jul 22 02:28:03.472006 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.471981 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9813117d-89f3-4796-848b-36b2b51fbdb4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9813117d-89f3-4796-848b-36b2b51fbdb4" (UID: "9813117d-89f3-4796-848b-36b2b51fbdb4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:28:03.473799 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.473777 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9813117d-89f3-4796-848b-36b2b51fbdb4-kube-api-access-ch6rp" (OuterVolumeSpecName: "kube-api-access-ch6rp") pod "9813117d-89f3-4796-848b-36b2b51fbdb4" (UID: "9813117d-89f3-4796-848b-36b2b51fbdb4"). InnerVolumeSpecName "kube-api-access-ch6rp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:28:03.473883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.473794 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2782815f-75b6-4619-82e6-f46a3dc11013-kube-api-access-cpvvk" (OuterVolumeSpecName: "kube-api-access-cpvvk") pod "2782815f-75b6-4619-82e6-f46a3dc11013" (UID: "2782815f-75b6-4619-82e6-f46a3dc11013"). InnerVolumeSpecName "kube-api-access-cpvvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:28:03.561334 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.561296 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" Jul 22 02:28:03.567371 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.567349 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fwz5b" Jul 22 02:28:03.572631 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.572581 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpvvk\" (UniqueName: \"kubernetes.io/projected/2782815f-75b6-4619-82e6-f46a3dc11013-kube-api-access-cpvvk\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:28:03.572631 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.572606 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9813117d-89f3-4796-848b-36b2b51fbdb4-extensions-socket-volume\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:28:03.572631 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.572622 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch6rp\" (UniqueName: \"kubernetes.io/projected/9813117d-89f3-4796-848b-36b2b51fbdb4-kube-api-access-ch6rp\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:28:03.611228 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.611183 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2782815f-75b6-4619-82e6-f46a3dc11013" path="/var/lib/kubelet/pods/2782815f-75b6-4619-82e6-f46a3dc11013/volumes" Jul 22 02:28:03.611955 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.611673 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9813117d-89f3-4796-848b-36b2b51fbdb4" path="/var/lib/kubelet/pods/9813117d-89f3-4796-848b-36b2b51fbdb4/volumes" Jul 22 02:28:03.657253 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.656900 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jxqwm" Jul 22 02:28:03.700183 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.700155 2557 generic.go:356] "Generic (PLEG): container finished" podID="9813117d-89f3-4796-848b-36b2b51fbdb4" containerID="24fb1a8c7653c0ff6e61a986868933ab78e7c508eacce498e510c8a4d944e3dd" exitCode=0 Jul 22 02:28:03.700349 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.700210 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qtxm7" Jul 22 02:28:03.700349 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.700259 2557 scope.go:117] "RemoveContainer" containerID="24fb1a8c7653c0ff6e61a986868933ab78e7c508eacce498e510c8a4d944e3dd" Jul 22 02:28:03.703077 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.702321 2557 generic.go:356] "Generic (PLEG): container finished" podID="2782815f-75b6-4619-82e6-f46a3dc11013" containerID="f514e7ec5db468028218a943f12edc73ec765fb2233eb4cd795629c666093cc1" exitCode=0 Jul 22 02:28:03.703077 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.702493 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59trn" Jul 22 02:28:03.712277 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.712253 2557 scope.go:117] "RemoveContainer" containerID="24fb1a8c7653c0ff6e61a986868933ab78e7c508eacce498e510c8a4d944e3dd" Jul 22 02:28:03.712567 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:28:03.712538 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24fb1a8c7653c0ff6e61a986868933ab78e7c508eacce498e510c8a4d944e3dd\": container with ID starting with 24fb1a8c7653c0ff6e61a986868933ab78e7c508eacce498e510c8a4d944e3dd not found: ID does not exist" containerID="24fb1a8c7653c0ff6e61a986868933ab78e7c508eacce498e510c8a4d944e3dd" Jul 22 02:28:03.712648 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.712590 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24fb1a8c7653c0ff6e61a986868933ab78e7c508eacce498e510c8a4d944e3dd"} err="failed to get container status \"24fb1a8c7653c0ff6e61a986868933ab78e7c508eacce498e510c8a4d944e3dd\": rpc error: code = NotFound desc = could not find container \"24fb1a8c7653c0ff6e61a986868933ab78e7c508eacce498e510c8a4d944e3dd\": container with ID starting with 24fb1a8c7653c0ff6e61a986868933ab78e7c508eacce498e510c8a4d944e3dd not found: ID does not exist" Jul 22 02:28:03.712648 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.712615 2557 scope.go:117] "RemoveContainer" containerID="f514e7ec5db468028218a943f12edc73ec765fb2233eb4cd795629c666093cc1" Jul 22 02:28:03.722491 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.722467 2557 scope.go:117] "RemoveContainer" containerID="f514e7ec5db468028218a943f12edc73ec765fb2233eb4cd795629c666093cc1" Jul 22 02:28:03.722810 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:28:03.722789 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f514e7ec5db468028218a943f12edc73ec765fb2233eb4cd795629c666093cc1\": container with ID starting with f514e7ec5db468028218a943f12edc73ec765fb2233eb4cd795629c666093cc1 not found: ID does not exist" containerID="f514e7ec5db468028218a943f12edc73ec765fb2233eb4cd795629c666093cc1" Jul 22 02:28:03.722877 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.722833 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f514e7ec5db468028218a943f12edc73ec765fb2233eb4cd795629c666093cc1"} err="failed to get container status \"f514e7ec5db468028218a943f12edc73ec765fb2233eb4cd795629c666093cc1\": rpc error: code = NotFound desc = could not find container \"f514e7ec5db468028218a943f12edc73ec765fb2233eb4cd795629c666093cc1\": container with ID starting with f514e7ec5db468028218a943f12edc73ec765fb2233eb4cd795629c666093cc1 not found: ID does not exist" Jul 22 02:28:03.737182 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.737156 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j"] Jul 22 02:28:03.740317 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:28:03.740288 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2602ef9a_1957_4cdd_9dd3_6891a77fa68f.slice/crio-9f24082a4ace4da18a6e4933d02722ccfca294057a3b4915f8f6a30b78d1e4c4 WatchSource:0}: Error finding container 9f24082a4ace4da18a6e4933d02722ccfca294057a3b4915f8f6a30b78d1e4c4: Status 404 returned error can't find the container with id 9f24082a4ace4da18a6e4933d02722ccfca294057a3b4915f8f6a30b78d1e4c4 Jul 22 02:28:03.761965 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:03.761945 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fwz5b"] Jul 22 02:28:03.767910 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:28:03.767884 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9db3f385_a143_4024_be8a_9d10507eff8a.slice/crio-575c4ab12f785a726aa5243b8d0a31ff8024bf1bf5e16bf6313560f6c756f956 WatchSource:0}: Error finding container 575c4ab12f785a726aa5243b8d0a31ff8024bf1bf5e16bf6313560f6c756f956: Status 404 returned error can't find the container with id 575c4ab12f785a726aa5243b8d0a31ff8024bf1bf5e16bf6313560f6c756f956 Jul 22 02:28:04.708101 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:04.708067 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fwz5b" event={"ID":"9db3f385-a143-4024-be8a-9d10507eff8a","Type":"ContainerStarted","Data":"26ee4c9e02ad003f1293e1ba565ce00cfd816787d680abc99f3ca86a67781745"} Jul 22 02:28:04.708518 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:04.708129 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fwz5b" Jul 22 02:28:04.708518 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:04.708155 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fwz5b" event={"ID":"9db3f385-a143-4024-be8a-9d10507eff8a","Type":"ContainerStarted","Data":"575c4ab12f785a726aa5243b8d0a31ff8024bf1bf5e16bf6313560f6c756f956"} Jul 22 02:28:04.711316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:04.711286 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" event={"ID":"2602ef9a-1957-4cdd-9dd3-6891a77fa68f","Type":"ContainerStarted","Data":"b7b8c58fd21b4c45a59594256e3e65c08966069e384c700e994ab5edc51d635a"} Jul 22 02:28:04.711433 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:04.711338 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" event={"ID":"2602ef9a-1957-4cdd-9dd3-6891a77fa68f","Type":"ContainerStarted","Data":"9f24082a4ace4da18a6e4933d02722ccfca294057a3b4915f8f6a30b78d1e4c4"} Jul 22 02:28:04.711433 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:04.711370 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" Jul 22 02:28:04.739527 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:04.739474 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fwz5b" podStartSLOduration=1.73945528 podStartE2EDuration="1.73945528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:04.739330035 +0000 UTC m=+515.648195032" watchObservedRunningTime="2026-07-22 02:28:04.73945528 +0000 UTC m=+515.648320277" Jul 22 02:28:05.543286 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:05.543232 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" podStartSLOduration=2.54321401 podStartE2EDuration="2.54321401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:04.812868775 +0000 UTC m=+515.721733771" watchObservedRunningTime="2026-07-22 02:28:05.54321401 +0000 UTC m=+516.452079005" Jul 22 02:28:05.544616 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:05.544599 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg"] Jul 22 02:28:05.547991 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:05.547977 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" Jul 22 02:28:05.583718 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:05.583690 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg"] Jul 22 02:28:05.588030 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:05.588003 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvjzc\" (UniqueName: \"kubernetes.io/projected/fab5c0a7-7259-45e5-b443-dd1887521b2e-kube-api-access-dvjzc\") pod \"kuadrant-operator-controller-manager-86878c769f-2stxg\" (UID: \"fab5c0a7-7259-45e5-b443-dd1887521b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" Jul 22 02:28:05.588165 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:05.588044 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fab5c0a7-7259-45e5-b443-dd1887521b2e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2stxg\" (UID: \"fab5c0a7-7259-45e5-b443-dd1887521b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" Jul 22 02:28:05.688426 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:05.688401 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvjzc\" (UniqueName: \"kubernetes.io/projected/fab5c0a7-7259-45e5-b443-dd1887521b2e-kube-api-access-dvjzc\") pod \"kuadrant-operator-controller-manager-86878c769f-2stxg\" (UID: \"fab5c0a7-7259-45e5-b443-dd1887521b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" Jul 22 02:28:05.688591 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:05.688438 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fab5c0a7-7259-45e5-b443-dd1887521b2e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2stxg\" (UID: \"fab5c0a7-7259-45e5-b443-dd1887521b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" Jul 22 02:28:05.688759 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:05.688743 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fab5c0a7-7259-45e5-b443-dd1887521b2e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2stxg\" (UID: \"fab5c0a7-7259-45e5-b443-dd1887521b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" Jul 22 02:28:05.700448 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:05.700413 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvjzc\" (UniqueName: \"kubernetes.io/projected/fab5c0a7-7259-45e5-b443-dd1887521b2e-kube-api-access-dvjzc\") pod \"kuadrant-operator-controller-manager-86878c769f-2stxg\" (UID: \"fab5c0a7-7259-45e5-b443-dd1887521b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" Jul 22 02:28:05.857808 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:05.857727 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" Jul 22 02:28:06.023968 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:06.023937 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg"] Jul 22 02:28:06.024315 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:28:06.024285 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfab5c0a7_7259_45e5_b443_dd1887521b2e.slice/crio-9cd46903381cb78bf76feb58aa382902fda39c4317e8d9179ab3f95bf097aa49 WatchSource:0}: Error finding container 9cd46903381cb78bf76feb58aa382902fda39c4317e8d9179ab3f95bf097aa49: Status 404 returned error can't find the container with id 9cd46903381cb78bf76feb58aa382902fda39c4317e8d9179ab3f95bf097aa49 Jul 22 02:28:06.722040 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:06.722006 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" event={"ID":"fab5c0a7-7259-45e5-b443-dd1887521b2e","Type":"ContainerStarted","Data":"9ec25c0b3e080efa1ad2e476eccf8579d80244bbec4bc318813159d117c8cd88"} Jul 22 02:28:06.722208 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:06.722057 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" event={"ID":"fab5c0a7-7259-45e5-b443-dd1887521b2e","Type":"ContainerStarted","Data":"9cd46903381cb78bf76feb58aa382902fda39c4317e8d9179ab3f95bf097aa49"} Jul 22 02:28:06.722208 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:06.722127 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" Jul 22 02:28:06.779755 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:06.779702 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" podStartSLOduration=1.779680911 podStartE2EDuration="1.779680911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:06.778645535 +0000 UTC m=+517.687510531" watchObservedRunningTime="2026-07-22 02:28:06.779680911 +0000 UTC m=+517.688545906" Jul 22 02:28:15.717700 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:15.717666 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" Jul 22 02:28:15.718148 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:15.717775 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fwz5b" Jul 22 02:28:17.728750 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:17.728718 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" Jul 22 02:28:17.797556 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:17.797523 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j"] Jul 22 02:28:17.797750 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:17.797710 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" podUID="2602ef9a-1957-4cdd-9dd3-6891a77fa68f" containerName="manager" containerID="cri-o://b7b8c58fd21b4c45a59594256e3e65c08966069e384c700e994ab5edc51d635a" gracePeriod=10 Jul 22 02:28:18.036743 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:18.036722 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" Jul 22 02:28:18.087695 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:18.087665 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2602ef9a-1957-4cdd-9dd3-6891a77fa68f-extensions-socket-volume\") pod \"2602ef9a-1957-4cdd-9dd3-6891a77fa68f\" (UID: \"2602ef9a-1957-4cdd-9dd3-6891a77fa68f\") " Jul 22 02:28:18.087880 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:18.087744 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkjg8\" (UniqueName: \"kubernetes.io/projected/2602ef9a-1957-4cdd-9dd3-6891a77fa68f-kube-api-access-wkjg8\") pod \"2602ef9a-1957-4cdd-9dd3-6891a77fa68f\" (UID: \"2602ef9a-1957-4cdd-9dd3-6891a77fa68f\") " Jul 22 02:28:18.088042 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:18.088021 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2602ef9a-1957-4cdd-9dd3-6891a77fa68f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2602ef9a-1957-4cdd-9dd3-6891a77fa68f" (UID: "2602ef9a-1957-4cdd-9dd3-6891a77fa68f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:28:18.089887 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:18.089863 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2602ef9a-1957-4cdd-9dd3-6891a77fa68f-kube-api-access-wkjg8" (OuterVolumeSpecName: "kube-api-access-wkjg8") pod "2602ef9a-1957-4cdd-9dd3-6891a77fa68f" (UID: "2602ef9a-1957-4cdd-9dd3-6891a77fa68f"). InnerVolumeSpecName "kube-api-access-wkjg8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:28:18.188666 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:18.188640 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2602ef9a-1957-4cdd-9dd3-6891a77fa68f-extensions-socket-volume\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:28:18.188666 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:18.188663 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkjg8\" (UniqueName: \"kubernetes.io/projected/2602ef9a-1957-4cdd-9dd3-6891a77fa68f-kube-api-access-wkjg8\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:28:18.779084 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:18.779050 2557 generic.go:356] "Generic (PLEG): container finished" podID="2602ef9a-1957-4cdd-9dd3-6891a77fa68f" containerID="b7b8c58fd21b4c45a59594256e3e65c08966069e384c700e994ab5edc51d635a" exitCode=0 Jul 22 02:28:18.779518 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:18.779110 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" Jul 22 02:28:18.779518 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:18.779143 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" event={"ID":"2602ef9a-1957-4cdd-9dd3-6891a77fa68f","Type":"ContainerDied","Data":"b7b8c58fd21b4c45a59594256e3e65c08966069e384c700e994ab5edc51d635a"} Jul 22 02:28:18.779518 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:18.779191 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j" event={"ID":"2602ef9a-1957-4cdd-9dd3-6891a77fa68f","Type":"ContainerDied","Data":"9f24082a4ace4da18a6e4933d02722ccfca294057a3b4915f8f6a30b78d1e4c4"} Jul 22 02:28:18.779518 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:18.779211 2557 scope.go:117] "RemoveContainer" containerID="b7b8c58fd21b4c45a59594256e3e65c08966069e384c700e994ab5edc51d635a" Jul 22 02:28:18.788349 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:18.788332 2557 scope.go:117] "RemoveContainer" containerID="b7b8c58fd21b4c45a59594256e3e65c08966069e384c700e994ab5edc51d635a" Jul 22 02:28:18.788605 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:28:18.788587 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7b8c58fd21b4c45a59594256e3e65c08966069e384c700e994ab5edc51d635a\": container with ID starting with b7b8c58fd21b4c45a59594256e3e65c08966069e384c700e994ab5edc51d635a not found: ID does not exist" containerID="b7b8c58fd21b4c45a59594256e3e65c08966069e384c700e994ab5edc51d635a" Jul 22 02:28:18.788681 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:18.788614 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7b8c58fd21b4c45a59594256e3e65c08966069e384c700e994ab5edc51d635a"} err="failed to get container status \"b7b8c58fd21b4c45a59594256e3e65c08966069e384c700e994ab5edc51d635a\": rpc error: code = NotFound desc = could not find container \"b7b8c58fd21b4c45a59594256e3e65c08966069e384c700e994ab5edc51d635a\": container with ID starting with b7b8c58fd21b4c45a59594256e3e65c08966069e384c700e994ab5edc51d635a not found: ID does not exist" Jul 22 02:28:18.809988 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:18.809962 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j"] Jul 22 02:28:18.824418 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:18.824395 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vxr7j"] Jul 22 02:28:19.608649 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:19.608618 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2602ef9a-1957-4cdd-9dd3-6891a77fa68f" path="/var/lib/kubelet/pods/2602ef9a-1957-4cdd-9dd3-6891a77fa68f/volumes" Jul 22 02:28:34.222194 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.222160 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg"] Jul 22 02:28:34.222720 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.222700 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2602ef9a-1957-4cdd-9dd3-6891a77fa68f" containerName="manager" Jul 22 02:28:34.222793 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.222720 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2602ef9a-1957-4cdd-9dd3-6891a77fa68f" containerName="manager" Jul 22 02:28:34.222865 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.222855 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2602ef9a-1957-4cdd-9dd3-6891a77fa68f" containerName="manager" Jul 22 02:28:34.225944 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.225924 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.229267 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.229249 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2bsf7\"" Jul 22 02:28:34.247203 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.247173 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg"] Jul 22 02:28:34.324591 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.324557 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.324591 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.324595 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.324890 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.324611 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wk5d\" (UniqueName: \"kubernetes.io/projected/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-kube-api-access-2wk5d\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.324890 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.324678 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.324890 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.324718 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.324890 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.324750 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.325037 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.324884 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.325037 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.324940 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.325037 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.324973 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.425671 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.425630 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.425896 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.425684 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.425896 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.425721 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.425896 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.425751 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.425896 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.425899 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.426240 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.425941 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.426240 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.425977 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wk5d\" (UniqueName: \"kubernetes.io/projected/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-kube-api-access-2wk5d\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.426240 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.426040 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.426240 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.426094 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.426447 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.426364 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.426447 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.426370 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.426447 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.426403 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.426763 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.426579 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.426763 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.426750 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.428246 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.428222 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.428649 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.428634 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.434678 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.434653 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.434863 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.434844 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wk5d\" (UniqueName: \"kubernetes.io/projected/42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6-kube-api-access-2wk5d\") pod \"maas-default-gateway-openshift-default-56654dcf7-hlntg\" (UID: \"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.535800 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.535717 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:34.684048 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.684024 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg"] Jul 22 02:28:34.686031 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:28:34.686002 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42dd0d7a_f7b4_4cec_b1f2_ee1a56b3a1d6.slice/crio-c34ffa53d9be6320b6d7227cd7cb70978402419f06bdd85c0976ad94c31295a5 WatchSource:0}: Error finding container c34ffa53d9be6320b6d7227cd7cb70978402419f06bdd85c0976ad94c31295a5: Status 404 returned error can't find the container with id c34ffa53d9be6320b6d7227cd7cb70978402419f06bdd85c0976ad94c31295a5 Jul 22 02:28:34.688091 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.688058 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:28:34.688189 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.688122 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:28:34.688189 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.688150 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:28:34.857220 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.857124 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" event={"ID":"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6","Type":"ContainerStarted","Data":"b94fb4cba0fb2d5c2adfd8252feb634aa52625f51f5da9fe93da5ef09b3e049c"} Jul 22 02:28:34.857220 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:34.857193 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" event={"ID":"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6","Type":"ContainerStarted","Data":"c34ffa53d9be6320b6d7227cd7cb70978402419f06bdd85c0976ad94c31295a5"} Jul 22 02:28:35.536054 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:35.536021 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:35.540626 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:35.540601 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:35.580304 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:35.580258 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" podStartSLOduration=1.580243282 podStartE2EDuration="1.580243282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:34.891197479 +0000 UTC m=+545.800062473" watchObservedRunningTime="2026-07-22 02:28:35.580243282 +0000 UTC m=+546.489108276" Jul 22 02:28:35.862024 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:35.861931 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:35.862857 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:35.862814 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:28:37.242783 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:37.242750 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-bxjwz"] Jul 22 02:28:37.245359 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:37.245335 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-bxjwz" Jul 22 02:28:37.252079 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:37.252057 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-htxd8\"" Jul 22 02:28:37.272797 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:37.272769 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-bxjwz"] Jul 22 02:28:37.352939 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:37.352905 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xtmh\" (UniqueName: \"kubernetes.io/projected/2a29a772-e074-4083-a410-9e57dd381ef0-kube-api-access-4xtmh\") pod \"authorino-58cd955675-bxjwz\" (UID: \"2a29a772-e074-4083-a410-9e57dd381ef0\") " pod="kuadrant-system/authorino-58cd955675-bxjwz" Jul 22 02:28:37.454342 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:37.454301 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xtmh\" (UniqueName: \"kubernetes.io/projected/2a29a772-e074-4083-a410-9e57dd381ef0-kube-api-access-4xtmh\") pod \"authorino-58cd955675-bxjwz\" (UID: \"2a29a772-e074-4083-a410-9e57dd381ef0\") " pod="kuadrant-system/authorino-58cd955675-bxjwz" Jul 22 02:28:37.468457 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:37.468425 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xtmh\" (UniqueName: \"kubernetes.io/projected/2a29a772-e074-4083-a410-9e57dd381ef0-kube-api-access-4xtmh\") pod \"authorino-58cd955675-bxjwz\" (UID: \"2a29a772-e074-4083-a410-9e57dd381ef0\") " pod="kuadrant-system/authorino-58cd955675-bxjwz" Jul 22 02:28:37.555445 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:37.555370 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-bxjwz" Jul 22 02:28:37.722782 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:37.722754 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-bxjwz"] Jul 22 02:28:37.725753 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:28:37.725725 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2a29a772_e074_4083_a410_9e57dd381ef0.slice/crio-77b114e26391894e00ad45ac36178295b7bcc97ef4c45dfdf1e1c3a22bb72a9c WatchSource:0}: Error finding container 77b114e26391894e00ad45ac36178295b7bcc97ef4c45dfdf1e1c3a22bb72a9c: Status 404 returned error can't find the container with id 77b114e26391894e00ad45ac36178295b7bcc97ef4c45dfdf1e1c3a22bb72a9c Jul 22 02:28:37.871843 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:37.871741 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-bxjwz" event={"ID":"2a29a772-e074-4083-a410-9e57dd381ef0","Type":"ContainerStarted","Data":"77b114e26391894e00ad45ac36178295b7bcc97ef4c45dfdf1e1c3a22bb72a9c"} Jul 22 02:28:40.887715 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:40.887680 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-bxjwz" event={"ID":"2a29a772-e074-4083-a410-9e57dd381ef0","Type":"ContainerStarted","Data":"58b7a15eee285581316555bf36e3ae359bb3dfad198ac36115197e10bed0dd7c"} Jul 22 02:28:40.906069 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:40.905963 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-bxjwz" podStartSLOduration=0.979523266 podStartE2EDuration="3.905944453s" totalImagesPullingTime="2.926421187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:40.905433637 +0000 UTC m=+551.814298632" watchObservedRunningTime="2026-07-22 02:28:40.905944453 +0000 UTC m=+551.814809449" Jul 22 02:28:42.610703 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:42.610673 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-bxjwz"] Jul 22 02:28:42.897776 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:42.897678 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-bxjwz" podUID="2a29a772-e074-4083-a410-9e57dd381ef0" containerName="authorino" containerID="cri-o://58b7a15eee285581316555bf36e3ae359bb3dfad198ac36115197e10bed0dd7c" gracePeriod=30 Jul 22 02:28:43.155240 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:43.155185 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-bxjwz" Jul 22 02:28:43.308130 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:43.308091 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xtmh\" (UniqueName: \"kubernetes.io/projected/2a29a772-e074-4083-a410-9e57dd381ef0-kube-api-access-4xtmh\") pod \"2a29a772-e074-4083-a410-9e57dd381ef0\" (UID: \"2a29a772-e074-4083-a410-9e57dd381ef0\") " Jul 22 02:28:43.310285 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:43.310252 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a29a772-e074-4083-a410-9e57dd381ef0-kube-api-access-4xtmh" (OuterVolumeSpecName: "kube-api-access-4xtmh") pod "2a29a772-e074-4083-a410-9e57dd381ef0" (UID: "2a29a772-e074-4083-a410-9e57dd381ef0"). InnerVolumeSpecName "kube-api-access-4xtmh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:28:43.409584 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:43.409484 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xtmh\" (UniqueName: \"kubernetes.io/projected/2a29a772-e074-4083-a410-9e57dd381ef0-kube-api-access-4xtmh\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:28:43.903569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:43.903528 2557 generic.go:356] "Generic (PLEG): container finished" podID="2a29a772-e074-4083-a410-9e57dd381ef0" containerID="58b7a15eee285581316555bf36e3ae359bb3dfad198ac36115197e10bed0dd7c" exitCode=0 Jul 22 02:28:43.903569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:43.903571 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-bxjwz" Jul 22 02:28:43.904124 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:43.903573 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-bxjwz" event={"ID":"2a29a772-e074-4083-a410-9e57dd381ef0","Type":"ContainerDied","Data":"58b7a15eee285581316555bf36e3ae359bb3dfad198ac36115197e10bed0dd7c"} Jul 22 02:28:43.904124 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:43.903617 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-bxjwz" event={"ID":"2a29a772-e074-4083-a410-9e57dd381ef0","Type":"ContainerDied","Data":"77b114e26391894e00ad45ac36178295b7bcc97ef4c45dfdf1e1c3a22bb72a9c"} Jul 22 02:28:43.904124 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:43.903635 2557 scope.go:117] "RemoveContainer" containerID="58b7a15eee285581316555bf36e3ae359bb3dfad198ac36115197e10bed0dd7c" Jul 22 02:28:43.911962 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:43.911944 2557 scope.go:117] "RemoveContainer" containerID="58b7a15eee285581316555bf36e3ae359bb3dfad198ac36115197e10bed0dd7c" Jul 22 02:28:43.912222 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:28:43.912205 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58b7a15eee285581316555bf36e3ae359bb3dfad198ac36115197e10bed0dd7c\": container with ID starting with 58b7a15eee285581316555bf36e3ae359bb3dfad198ac36115197e10bed0dd7c not found: ID does not exist" containerID="58b7a15eee285581316555bf36e3ae359bb3dfad198ac36115197e10bed0dd7c" Jul 22 02:28:43.912272 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:43.912231 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58b7a15eee285581316555bf36e3ae359bb3dfad198ac36115197e10bed0dd7c"} err="failed to get container status \"58b7a15eee285581316555bf36e3ae359bb3dfad198ac36115197e10bed0dd7c\": rpc error: code = NotFound desc = could not find container \"58b7a15eee285581316555bf36e3ae359bb3dfad198ac36115197e10bed0dd7c\": container with ID starting with 58b7a15eee285581316555bf36e3ae359bb3dfad198ac36115197e10bed0dd7c not found: ID does not exist" Jul 22 02:28:43.921468 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:43.921445 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-bxjwz"] Jul 22 02:28:43.924966 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:43.924934 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-bxjwz"] Jul 22 02:28:45.608408 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:28:45.608365 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a29a772-e074-4083-a410-9e57dd381ef0" path="/var/lib/kubelet/pods/2a29a772-e074-4083-a410-9e57dd381ef0/volumes" Jul 22 02:29:33.151837 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.151790 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4bd42"] Jul 22 02:29:33.152412 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.152163 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a29a772-e074-4083-a410-9e57dd381ef0" containerName="authorino" Jul 22 02:29:33.152412 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.152177 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a29a772-e074-4083-a410-9e57dd381ef0" containerName="authorino" Jul 22 02:29:33.152412 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.152249 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a29a772-e074-4083-a410-9e57dd381ef0" containerName="authorino" Jul 22 02:29:33.155083 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.155066 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4bd42" Jul 22 02:29:33.158046 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.158025 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-htxd8\"" Jul 22 02:29:33.162857 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.162808 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4bd42"] Jul 22 02:29:33.211066 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.211040 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsvrt\" (UniqueName: \"kubernetes.io/projected/38019092-8ebd-4ada-be54-1a4335ac4496-kube-api-access-tsvrt\") pod \"authorino-5d4b67c446-4bd42\" (UID: \"38019092-8ebd-4ada-be54-1a4335ac4496\") " pod="kuadrant-system/authorino-5d4b67c446-4bd42" Jul 22 02:29:33.311715 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.311682 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsvrt\" (UniqueName: \"kubernetes.io/projected/38019092-8ebd-4ada-be54-1a4335ac4496-kube-api-access-tsvrt\") pod \"authorino-5d4b67c446-4bd42\" (UID: \"38019092-8ebd-4ada-be54-1a4335ac4496\") " pod="kuadrant-system/authorino-5d4b67c446-4bd42" Jul 22 02:29:33.320216 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.320191 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsvrt\" (UniqueName: \"kubernetes.io/projected/38019092-8ebd-4ada-be54-1a4335ac4496-kube-api-access-tsvrt\") pod \"authorino-5d4b67c446-4bd42\" (UID: \"38019092-8ebd-4ada-be54-1a4335ac4496\") " pod="kuadrant-system/authorino-5d4b67c446-4bd42" Jul 22 02:29:33.401765 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.401740 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4bd42"] Jul 22 02:29:33.402024 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.401988 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4bd42" Jul 22 02:29:33.427465 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.427435 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56d46ddb99-7cd8b"] Jul 22 02:29:33.430041 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.430023 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56d46ddb99-7cd8b" Jul 22 02:29:33.440623 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.440598 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56d46ddb99-7cd8b"] Jul 22 02:29:33.512625 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.512595 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lkmq\" (UniqueName: \"kubernetes.io/projected/f9ed2348-a165-4d32-b8de-0e843d9d6674-kube-api-access-4lkmq\") pod \"authorino-56d46ddb99-7cd8b\" (UID: \"f9ed2348-a165-4d32-b8de-0e843d9d6674\") " pod="kuadrant-system/authorino-56d46ddb99-7cd8b" Jul 22 02:29:33.536102 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.536076 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4bd42"] Jul 22 02:29:33.538419 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:29:33.538393 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38019092_8ebd_4ada_be54_1a4335ac4496.slice/crio-2e06d07d86ce8130821c1524231981f6358daddc7c7b76dfde864219a7bae7bb WatchSource:0}: Error finding container 2e06d07d86ce8130821c1524231981f6358daddc7c7b76dfde864219a7bae7bb: Status 404 returned error can't find the container with id 2e06d07d86ce8130821c1524231981f6358daddc7c7b76dfde864219a7bae7bb Jul 22 02:29:33.613744 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.613711 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lkmq\" (UniqueName: \"kubernetes.io/projected/f9ed2348-a165-4d32-b8de-0e843d9d6674-kube-api-access-4lkmq\") pod \"authorino-56d46ddb99-7cd8b\" (UID: \"f9ed2348-a165-4d32-b8de-0e843d9d6674\") " pod="kuadrant-system/authorino-56d46ddb99-7cd8b" Jul 22 02:29:33.622521 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.622499 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lkmq\" (UniqueName: \"kubernetes.io/projected/f9ed2348-a165-4d32-b8de-0e843d9d6674-kube-api-access-4lkmq\") pod \"authorino-56d46ddb99-7cd8b\" (UID: \"f9ed2348-a165-4d32-b8de-0e843d9d6674\") " pod="kuadrant-system/authorino-56d46ddb99-7cd8b" Jul 22 02:29:33.698174 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.698100 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56d46ddb99-7cd8b"] Jul 22 02:29:33.698351 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.698340 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56d46ddb99-7cd8b" Jul 22 02:29:33.828704 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:33.828646 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56d46ddb99-7cd8b"] Jul 22 02:29:33.831395 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:29:33.831359 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9ed2348_a165_4d32_b8de_0e843d9d6674.slice/crio-7d33858f627431e011e2d4d5b6be6783676d1291734a73466a3847adea8ca252 WatchSource:0}: Error finding container 7d33858f627431e011e2d4d5b6be6783676d1291734a73466a3847adea8ca252: Status 404 returned error can't find the container with id 7d33858f627431e011e2d4d5b6be6783676d1291734a73466a3847adea8ca252 Jul 22 02:29:34.126495 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:34.126450 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56d46ddb99-7cd8b" event={"ID":"f9ed2348-a165-4d32-b8de-0e843d9d6674","Type":"ContainerStarted","Data":"7d33858f627431e011e2d4d5b6be6783676d1291734a73466a3847adea8ca252"} Jul 22 02:29:34.128019 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:34.127991 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4bd42" event={"ID":"38019092-8ebd-4ada-be54-1a4335ac4496","Type":"ContainerStarted","Data":"36332d67815a8a876a9912ca2a0d372b4764fa9e8cc9c00cb189a506be11533e"} Jul 22 02:29:34.128153 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:34.128032 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4bd42" event={"ID":"38019092-8ebd-4ada-be54-1a4335ac4496","Type":"ContainerStarted","Data":"2e06d07d86ce8130821c1524231981f6358daddc7c7b76dfde864219a7bae7bb"} Jul 22 02:29:34.128153 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:34.128053 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-4bd42" podUID="38019092-8ebd-4ada-be54-1a4335ac4496" containerName="authorino" containerID="cri-o://36332d67815a8a876a9912ca2a0d372b4764fa9e8cc9c00cb189a506be11533e" gracePeriod=30 Jul 22 02:29:34.145621 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:34.145579 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-4bd42" podStartSLOduration=0.78550067 podStartE2EDuration="1.145561787s" totalImagesPullingTime="360.061117ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:29:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:34.143975167 +0000 UTC m=+605.052840162" watchObservedRunningTime="2026-07-22 02:29:34.145561787 +0000 UTC m=+605.054426782" Jul 22 02:29:34.388835 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:34.388796 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4bd42" Jul 22 02:29:34.521568 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:34.521488 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsvrt\" (UniqueName: \"kubernetes.io/projected/38019092-8ebd-4ada-be54-1a4335ac4496-kube-api-access-tsvrt\") pod \"38019092-8ebd-4ada-be54-1a4335ac4496\" (UID: \"38019092-8ebd-4ada-be54-1a4335ac4496\") " Jul 22 02:29:34.523772 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:34.523748 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38019092-8ebd-4ada-be54-1a4335ac4496-kube-api-access-tsvrt" (OuterVolumeSpecName: "kube-api-access-tsvrt") pod "38019092-8ebd-4ada-be54-1a4335ac4496" (UID: "38019092-8ebd-4ada-be54-1a4335ac4496"). InnerVolumeSpecName "kube-api-access-tsvrt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:29:34.622187 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:34.622150 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsvrt\" (UniqueName: \"kubernetes.io/projected/38019092-8ebd-4ada-be54-1a4335ac4496-kube-api-access-tsvrt\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:29:35.134480 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:35.134438 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56d46ddb99-7cd8b" event={"ID":"f9ed2348-a165-4d32-b8de-0e843d9d6674","Type":"ContainerStarted","Data":"30bfd97efbd02f5f85d8bb8f90d89efb015208994bceb5e39e6e4fd290b31a22"} Jul 22 02:29:35.134665 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:35.134523 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-56d46ddb99-7cd8b" podUID="f9ed2348-a165-4d32-b8de-0e843d9d6674" containerName="authorino" containerID="cri-o://30bfd97efbd02f5f85d8bb8f90d89efb015208994bceb5e39e6e4fd290b31a22" gracePeriod=30 Jul 22 02:29:35.135705 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:35.135676 2557 generic.go:356] "Generic (PLEG): container finished" podID="38019092-8ebd-4ada-be54-1a4335ac4496" containerID="36332d67815a8a876a9912ca2a0d372b4764fa9e8cc9c00cb189a506be11533e" exitCode=0 Jul 22 02:29:35.135788 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:35.135715 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4bd42" Jul 22 02:29:35.135788 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:35.135761 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4bd42" event={"ID":"38019092-8ebd-4ada-be54-1a4335ac4496","Type":"ContainerDied","Data":"36332d67815a8a876a9912ca2a0d372b4764fa9e8cc9c00cb189a506be11533e"} Jul 22 02:29:35.135882 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:35.135805 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4bd42" event={"ID":"38019092-8ebd-4ada-be54-1a4335ac4496","Type":"ContainerDied","Data":"2e06d07d86ce8130821c1524231981f6358daddc7c7b76dfde864219a7bae7bb"} Jul 22 02:29:35.135882 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:35.135840 2557 scope.go:117] "RemoveContainer" containerID="36332d67815a8a876a9912ca2a0d372b4764fa9e8cc9c00cb189a506be11533e" Jul 22 02:29:35.144711 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:35.144693 2557 scope.go:117] "RemoveContainer" containerID="36332d67815a8a876a9912ca2a0d372b4764fa9e8cc9c00cb189a506be11533e" Jul 22 02:29:35.145072 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:29:35.145052 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36332d67815a8a876a9912ca2a0d372b4764fa9e8cc9c00cb189a506be11533e\": container with ID starting with 36332d67815a8a876a9912ca2a0d372b4764fa9e8cc9c00cb189a506be11533e not found: ID does not exist" containerID="36332d67815a8a876a9912ca2a0d372b4764fa9e8cc9c00cb189a506be11533e" Jul 22 02:29:35.145157 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:35.145082 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36332d67815a8a876a9912ca2a0d372b4764fa9e8cc9c00cb189a506be11533e"} err="failed to get container status \"36332d67815a8a876a9912ca2a0d372b4764fa9e8cc9c00cb189a506be11533e\": rpc error: code = NotFound desc = could not find container \"36332d67815a8a876a9912ca2a0d372b4764fa9e8cc9c00cb189a506be11533e\": container with ID starting with 36332d67815a8a876a9912ca2a0d372b4764fa9e8cc9c00cb189a506be11533e not found: ID does not exist" Jul 22 02:29:35.150506 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:35.150468 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-56d46ddb99-7cd8b" podStartSLOduration=1.811305717 podStartE2EDuration="2.150453167s" totalImagesPullingTime="339.14745ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:29:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:35.149139447 +0000 UTC m=+606.058004443" watchObservedRunningTime="2026-07-22 02:29:35.150453167 +0000 UTC m=+606.059318218" Jul 22 02:29:35.163512 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:35.163489 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4bd42"] Jul 22 02:29:35.168492 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:35.168471 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4bd42"] Jul 22 02:29:35.378873 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:35.378848 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56d46ddb99-7cd8b" Jul 22 02:29:35.428339 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:35.428263 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lkmq\" (UniqueName: \"kubernetes.io/projected/f9ed2348-a165-4d32-b8de-0e843d9d6674-kube-api-access-4lkmq\") pod \"f9ed2348-a165-4d32-b8de-0e843d9d6674\" (UID: \"f9ed2348-a165-4d32-b8de-0e843d9d6674\") " Jul 22 02:29:35.430355 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:35.430333 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9ed2348-a165-4d32-b8de-0e843d9d6674-kube-api-access-4lkmq" (OuterVolumeSpecName: "kube-api-access-4lkmq") pod "f9ed2348-a165-4d32-b8de-0e843d9d6674" (UID: "f9ed2348-a165-4d32-b8de-0e843d9d6674"). InnerVolumeSpecName "kube-api-access-4lkmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:29:35.529350 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:35.529319 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lkmq\" (UniqueName: \"kubernetes.io/projected/f9ed2348-a165-4d32-b8de-0e843d9d6674-kube-api-access-4lkmq\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:29:35.608314 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:35.608285 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38019092-8ebd-4ada-be54-1a4335ac4496" path="/var/lib/kubelet/pods/38019092-8ebd-4ada-be54-1a4335ac4496/volumes" Jul 22 02:29:36.141099 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:36.141061 2557 generic.go:356] "Generic (PLEG): container finished" podID="f9ed2348-a165-4d32-b8de-0e843d9d6674" containerID="30bfd97efbd02f5f85d8bb8f90d89efb015208994bceb5e39e6e4fd290b31a22" exitCode=0 Jul 22 02:29:36.141291 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:36.141109 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56d46ddb99-7cd8b" Jul 22 02:29:36.141291 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:36.141129 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56d46ddb99-7cd8b" event={"ID":"f9ed2348-a165-4d32-b8de-0e843d9d6674","Type":"ContainerDied","Data":"30bfd97efbd02f5f85d8bb8f90d89efb015208994bceb5e39e6e4fd290b31a22"} Jul 22 02:29:36.141291 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:36.141171 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56d46ddb99-7cd8b" event={"ID":"f9ed2348-a165-4d32-b8de-0e843d9d6674","Type":"ContainerDied","Data":"7d33858f627431e011e2d4d5b6be6783676d1291734a73466a3847adea8ca252"} Jul 22 02:29:36.141291 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:36.141192 2557 scope.go:117] "RemoveContainer" containerID="30bfd97efbd02f5f85d8bb8f90d89efb015208994bceb5e39e6e4fd290b31a22" Jul 22 02:29:36.149137 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:36.149118 2557 scope.go:117] "RemoveContainer" containerID="30bfd97efbd02f5f85d8bb8f90d89efb015208994bceb5e39e6e4fd290b31a22" Jul 22 02:29:36.149391 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:29:36.149371 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30bfd97efbd02f5f85d8bb8f90d89efb015208994bceb5e39e6e4fd290b31a22\": container with ID starting with 30bfd97efbd02f5f85d8bb8f90d89efb015208994bceb5e39e6e4fd290b31a22 not found: ID does not exist" containerID="30bfd97efbd02f5f85d8bb8f90d89efb015208994bceb5e39e6e4fd290b31a22" Jul 22 02:29:36.149454 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:36.149403 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30bfd97efbd02f5f85d8bb8f90d89efb015208994bceb5e39e6e4fd290b31a22"} err="failed to get container status \"30bfd97efbd02f5f85d8bb8f90d89efb015208994bceb5e39e6e4fd290b31a22\": rpc error: code = NotFound desc = could not find container \"30bfd97efbd02f5f85d8bb8f90d89efb015208994bceb5e39e6e4fd290b31a22\": container with ID starting with 30bfd97efbd02f5f85d8bb8f90d89efb015208994bceb5e39e6e4fd290b31a22 not found: ID does not exist" Jul 22 02:29:36.190439 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:36.190409 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56d46ddb99-7cd8b"] Jul 22 02:29:36.193376 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:36.193357 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56d46ddb99-7cd8b"] Jul 22 02:29:37.609411 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:29:37.609376 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9ed2348-a165-4d32-b8de-0e843d9d6674" path="/var/lib/kubelet/pods/f9ed2348-a165-4d32-b8de-0e843d9d6674/volumes" Jul 22 02:30:25.360258 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:25.360223 2557 generic.go:356] "Generic (PLEG): container finished" podID="42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6" containerID="b94fb4cba0fb2d5c2adfd8252feb634aa52625f51f5da9fe93da5ef09b3e049c" exitCode=0 Jul 22 02:30:25.360682 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:25.360299 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" event={"ID":"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6","Type":"ContainerDied","Data":"b94fb4cba0fb2d5c2adfd8252feb634aa52625f51f5da9fe93da5ef09b3e049c"} Jul 22 02:30:25.360682 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:25.360674 2557 scope.go:117] "RemoveContainer" containerID="b94fb4cba0fb2d5c2adfd8252feb634aa52625f51f5da9fe93da5ef09b3e049c" Jul 22 02:30:25.361702 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:25.361668 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:30:25.361777 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:25.361760 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:30:25.361856 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:25.361814 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:30:26.366970 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:26.366940 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" event={"ID":"42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6","Type":"ContainerStarted","Data":"9cbb6bc0aab1e45cc8500b152d6b325c2b52beeefe413c9fd7744e9d0c5f8737"} Jul 22 02:30:26.536440 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:26.536397 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:30:26.536681 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:26.536657 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hlntg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 22 02:30:26.536767 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:26.536716 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" podUID="42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 22 02:30:27.536722 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:27.536686 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hlntg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 22 02:30:27.537117 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:27.536738 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" podUID="42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 22 02:30:28.536278 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:28.536249 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hlntg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 22 02:30:28.536443 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:28.536302 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" podUID="42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 22 02:30:29.537096 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:29.537056 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hlntg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 22 02:30:29.537540 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:29.537126 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" podUID="42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 22 02:30:30.540113 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:30.540082 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:30:30.540504 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:30.540330 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:30:30.540989 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:30.540971 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hlntg" Jul 22 02:30:41.052444 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.052409 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5"] Jul 22 02:30:41.053056 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.053033 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38019092-8ebd-4ada-be54-1a4335ac4496" containerName="authorino" Jul 22 02:30:41.053155 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.053057 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="38019092-8ebd-4ada-be54-1a4335ac4496" containerName="authorino" Jul 22 02:30:41.053155 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.053095 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9ed2348-a165-4d32-b8de-0e843d9d6674" containerName="authorino" Jul 22 02:30:41.053155 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.053105 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9ed2348-a165-4d32-b8de-0e843d9d6674" containerName="authorino" Jul 22 02:30:41.053305 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.053196 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9ed2348-a165-4d32-b8de-0e843d9d6674" containerName="authorino" Jul 22 02:30:41.053305 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.053212 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="38019092-8ebd-4ada-be54-1a4335ac4496" containerName="authorino" Jul 22 02:30:41.056126 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.056104 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.060538 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.060517 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 22 02:30:41.060645 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.060624 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4mjch\"" Jul 22 02:30:41.060713 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.060520 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 02:30:41.060713 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.060555 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 02:30:41.070549 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.070528 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5"] Jul 22 02:30:41.199764 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.199731 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/078c49d1-ecca-46d4-81dd-4071bb34a763-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.199946 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.199778 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/078c49d1-ecca-46d4-81dd-4071bb34a763-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.199946 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.199888 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/078c49d1-ecca-46d4-81dd-4071bb34a763-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.199946 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.199928 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/078c49d1-ecca-46d4-81dd-4071bb34a763-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.200080 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.199950 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq7p8\" (UniqueName: \"kubernetes.io/projected/078c49d1-ecca-46d4-81dd-4071bb34a763-kube-api-access-cq7p8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.200080 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.199973 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/078c49d1-ecca-46d4-81dd-4071bb34a763-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.301133 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.301095 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/078c49d1-ecca-46d4-81dd-4071bb34a763-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.301329 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.301143 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/078c49d1-ecca-46d4-81dd-4071bb34a763-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.301329 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.301163 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq7p8\" (UniqueName: \"kubernetes.io/projected/078c49d1-ecca-46d4-81dd-4071bb34a763-kube-api-access-cq7p8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.301329 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.301184 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/078c49d1-ecca-46d4-81dd-4071bb34a763-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.301329 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.301213 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/078c49d1-ecca-46d4-81dd-4071bb34a763-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.301329 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.301265 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/078c49d1-ecca-46d4-81dd-4071bb34a763-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.301606 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.301558 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/078c49d1-ecca-46d4-81dd-4071bb34a763-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.301606 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.301599 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/078c49d1-ecca-46d4-81dd-4071bb34a763-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.301714 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.301654 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/078c49d1-ecca-46d4-81dd-4071bb34a763-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.303654 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.303604 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/078c49d1-ecca-46d4-81dd-4071bb34a763-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.303769 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.303753 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/078c49d1-ecca-46d4-81dd-4071bb34a763-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.311117 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.311084 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq7p8\" (UniqueName: \"kubernetes.io/projected/078c49d1-ecca-46d4-81dd-4071bb34a763-kube-api-access-cq7p8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5\" (UID: \"078c49d1-ecca-46d4-81dd-4071bb34a763\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.367021 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.366988 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:41.532556 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.532524 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5"] Jul 22 02:30:41.533429 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:30:41.533403 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod078c49d1_ecca_46d4_81dd_4071bb34a763.slice/crio-f72c6ae108e503446f477a27d447d3c3c26ffd4fdf15d8bcba2b9c74bcadd00b WatchSource:0}: Error finding container f72c6ae108e503446f477a27d447d3c3c26ffd4fdf15d8bcba2b9c74bcadd00b: Status 404 returned error can't find the container with id f72c6ae108e503446f477a27d447d3c3c26ffd4fdf15d8bcba2b9c74bcadd00b Jul 22 02:30:41.535011 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:41.534991 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:30:42.450181 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:42.450143 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" event={"ID":"078c49d1-ecca-46d4-81dd-4071bb34a763","Type":"ContainerStarted","Data":"f72c6ae108e503446f477a27d447d3c3c26ffd4fdf15d8bcba2b9c74bcadd00b"} Jul 22 02:30:43.456331 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:43.456294 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" event={"ID":"078c49d1-ecca-46d4-81dd-4071bb34a763","Type":"ContainerStarted","Data":"2b7d76e88afba99878719b5ab0352c1a55ed3175339dedae8f1bbf20e5d8ac72"} Jul 22 02:30:43.478171 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:43.478126 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" podStartSLOduration=1.464566556 podStartE2EDuration="2.478111912s" totalImagesPullingTime="1.013545356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:30:43.477461216 +0000 UTC m=+674.386326212" watchObservedRunningTime="2026-07-22 02:30:43.478111912 +0000 UTC m=+674.386976907" Jul 22 02:30:51.367661 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:51.367608 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:51.367661 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:51.367680 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:51.380223 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:51.380198 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:30:51.504491 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:30:51.504465 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5" Jul 22 02:31:10.245414 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.245333 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p"] Jul 22 02:31:10.249120 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.249097 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.252633 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.252610 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 22 02:31:10.262317 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.262293 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p"] Jul 22 02:31:10.359668 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.359634 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ff43abe-5ba8-4077-9907-9a1aab83aebb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.359668 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.359667 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwhv9\" (UniqueName: \"kubernetes.io/projected/2ff43abe-5ba8-4077-9907-9a1aab83aebb-kube-api-access-mwhv9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.359916 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.359723 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ff43abe-5ba8-4077-9907-9a1aab83aebb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.359916 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.359760 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ff43abe-5ba8-4077-9907-9a1aab83aebb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.359916 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.359775 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ff43abe-5ba8-4077-9907-9a1aab83aebb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.359916 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.359797 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ff43abe-5ba8-4077-9907-9a1aab83aebb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.461143 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.461096 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ff43abe-5ba8-4077-9907-9a1aab83aebb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.461541 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.461169 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ff43abe-5ba8-4077-9907-9a1aab83aebb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.461541 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.461192 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ff43abe-5ba8-4077-9907-9a1aab83aebb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.461541 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.461210 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ff43abe-5ba8-4077-9907-9a1aab83aebb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.461541 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.461241 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ff43abe-5ba8-4077-9907-9a1aab83aebb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.461541 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.461265 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwhv9\" (UniqueName: \"kubernetes.io/projected/2ff43abe-5ba8-4077-9907-9a1aab83aebb-kube-api-access-mwhv9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.461802 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.461569 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ff43abe-5ba8-4077-9907-9a1aab83aebb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.461802 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.461582 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ff43abe-5ba8-4077-9907-9a1aab83aebb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.461802 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.461691 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ff43abe-5ba8-4077-9907-9a1aab83aebb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.463720 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.463697 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ff43abe-5ba8-4077-9907-9a1aab83aebb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.464113 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.464090 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ff43abe-5ba8-4077-9907-9a1aab83aebb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.470232 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.470206 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwhv9\" (UniqueName: \"kubernetes.io/projected/2ff43abe-5ba8-4077-9907-9a1aab83aebb-kube-api-access-mwhv9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x9q7p\" (UID: \"2ff43abe-5ba8-4077-9907-9a1aab83aebb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.560531 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.560438 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:10.701315 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:10.701289 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p"] Jul 22 02:31:10.703380 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:31:10.703351 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ff43abe_5ba8_4077_9907_9a1aab83aebb.slice/crio-699df6a48f28d8609695794fb829f51aa082459bde1ea9d4ec91f9bc2dbe9afd WatchSource:0}: Error finding container 699df6a48f28d8609695794fb829f51aa082459bde1ea9d4ec91f9bc2dbe9afd: Status 404 returned error can't find the container with id 699df6a48f28d8609695794fb829f51aa082459bde1ea9d4ec91f9bc2dbe9afd Jul 22 02:31:11.585160 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:11.585124 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" event={"ID":"2ff43abe-5ba8-4077-9907-9a1aab83aebb","Type":"ContainerStarted","Data":"a58f58b2126de1cf9da6eb2510127a17f236aeaab96b3cb47fe804c85705adf3"} Jul 22 02:31:11.585646 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:11.585186 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" event={"ID":"2ff43abe-5ba8-4077-9907-9a1aab83aebb","Type":"ContainerStarted","Data":"699df6a48f28d8609695794fb829f51aa082459bde1ea9d4ec91f9bc2dbe9afd"} Jul 22 02:31:20.561551 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:20.561509 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:20.562105 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:20.561856 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:20.574467 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:20.574440 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:31:20.597037 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:20.596971 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" podStartSLOduration=10.406541134 podStartE2EDuration="10.596953519s" totalImagesPullingTime="190.412385ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:31:11.654887943 +0000 UTC m=+702.563752938" watchObservedRunningTime="2026-07-22 02:31:20.596953519 +0000 UTC m=+711.505818514" Jul 22 02:31:20.635530 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:31:20.635498 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x9q7p" Jul 22 02:32:34.159741 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:32:34.159708 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7dc4cf4958-8f6qx"] Jul 22 02:32:34.162952 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:32:34.162935 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dc4cf4958-8f6qx" Jul 22 02:32:34.167026 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:32:34.167006 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 22 02:32:34.167026 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:32:34.167013 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-htxd8\"" Jul 22 02:32:34.172085 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:32:34.172049 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7dc4cf4958-8f6qx"] Jul 22 02:32:34.290542 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:32:34.290499 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bb5c5554-0112-4399-ac9f-bb3f421258e9-tls-cert\") pod \"authorino-7dc4cf4958-8f6qx\" (UID: \"bb5c5554-0112-4399-ac9f-bb3f421258e9\") " pod="kuadrant-system/authorino-7dc4cf4958-8f6qx" Jul 22 02:32:34.290687 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:32:34.290615 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpb42\" (UniqueName: \"kubernetes.io/projected/bb5c5554-0112-4399-ac9f-bb3f421258e9-kube-api-access-xpb42\") pod \"authorino-7dc4cf4958-8f6qx\" (UID: \"bb5c5554-0112-4399-ac9f-bb3f421258e9\") " pod="kuadrant-system/authorino-7dc4cf4958-8f6qx" Jul 22 02:32:34.391326 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:32:34.391287 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bb5c5554-0112-4399-ac9f-bb3f421258e9-tls-cert\") pod \"authorino-7dc4cf4958-8f6qx\" (UID: \"bb5c5554-0112-4399-ac9f-bb3f421258e9\") " pod="kuadrant-system/authorino-7dc4cf4958-8f6qx" Jul 22 02:32:34.391479 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:32:34.391386 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpb42\" (UniqueName: \"kubernetes.io/projected/bb5c5554-0112-4399-ac9f-bb3f421258e9-kube-api-access-xpb42\") pod \"authorino-7dc4cf4958-8f6qx\" (UID: \"bb5c5554-0112-4399-ac9f-bb3f421258e9\") " pod="kuadrant-system/authorino-7dc4cf4958-8f6qx" Jul 22 02:32:34.393897 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:32:34.393866 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bb5c5554-0112-4399-ac9f-bb3f421258e9-tls-cert\") pod \"authorino-7dc4cf4958-8f6qx\" (UID: \"bb5c5554-0112-4399-ac9f-bb3f421258e9\") " pod="kuadrant-system/authorino-7dc4cf4958-8f6qx" Jul 22 02:32:34.400146 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:32:34.400122 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpb42\" (UniqueName: \"kubernetes.io/projected/bb5c5554-0112-4399-ac9f-bb3f421258e9-kube-api-access-xpb42\") pod \"authorino-7dc4cf4958-8f6qx\" (UID: \"bb5c5554-0112-4399-ac9f-bb3f421258e9\") " pod="kuadrant-system/authorino-7dc4cf4958-8f6qx" Jul 22 02:32:34.472932 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:32:34.472900 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dc4cf4958-8f6qx" Jul 22 02:32:34.624691 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:32:34.624669 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7dc4cf4958-8f6qx"] Jul 22 02:32:34.962541 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:32:34.962506 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7dc4cf4958-8f6qx" event={"ID":"bb5c5554-0112-4399-ac9f-bb3f421258e9","Type":"ContainerStarted","Data":"45dbbb91785c7f6205d024e4be936412ec0627d918e494d625ef9aa9885e1352"} Jul 22 02:32:35.971715 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:32:35.971676 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7dc4cf4958-8f6qx" event={"ID":"bb5c5554-0112-4399-ac9f-bb3f421258e9","Type":"ContainerStarted","Data":"46d6057585e2d64cbf4237d4162f703994668dc7966e8e32f1b5e6cf3f169620"} Jul 22 02:32:35.991946 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:32:35.991896 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7dc4cf4958-8f6qx" podStartSLOduration=1.491933751 podStartE2EDuration="1.991877358s" totalImagesPullingTime="499.943607ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:32:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:32:35.989669642 +0000 UTC m=+786.898534653" watchObservedRunningTime="2026-07-22 02:32:35.991877358 +0000 UTC m=+786.900742353" Jul 22 02:39:28.160708 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:28.160668 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg"] Jul 22 02:39:28.161330 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:28.160971 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" podUID="fab5c0a7-7259-45e5-b443-dd1887521b2e" containerName="manager" containerID="cri-o://9ec25c0b3e080efa1ad2e476eccf8579d80244bbec4bc318813159d117c8cd88" gracePeriod=10 Jul 22 02:39:31.920369 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:31.920343 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" Jul 22 02:39:31.927562 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:31.927535 2557 generic.go:356] "Generic (PLEG): container finished" podID="fab5c0a7-7259-45e5-b443-dd1887521b2e" containerID="9ec25c0b3e080efa1ad2e476eccf8579d80244bbec4bc318813159d117c8cd88" exitCode=0 Jul 22 02:39:31.927685 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:31.927575 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" Jul 22 02:39:31.927685 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:31.927621 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" event={"ID":"fab5c0a7-7259-45e5-b443-dd1887521b2e","Type":"ContainerDied","Data":"9ec25c0b3e080efa1ad2e476eccf8579d80244bbec4bc318813159d117c8cd88"} Jul 22 02:39:31.927685 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:31.927675 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg" event={"ID":"fab5c0a7-7259-45e5-b443-dd1887521b2e","Type":"ContainerDied","Data":"9cd46903381cb78bf76feb58aa382902fda39c4317e8d9179ab3f95bf097aa49"} Jul 22 02:39:31.927784 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:31.927698 2557 scope.go:117] "RemoveContainer" containerID="9ec25c0b3e080efa1ad2e476eccf8579d80244bbec4bc318813159d117c8cd88" Jul 22 02:39:31.937397 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:31.937379 2557 scope.go:117] "RemoveContainer" containerID="9ec25c0b3e080efa1ad2e476eccf8579d80244bbec4bc318813159d117c8cd88" Jul 22 02:39:31.937759 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:39:31.937690 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ec25c0b3e080efa1ad2e476eccf8579d80244bbec4bc318813159d117c8cd88\": container with ID starting with 9ec25c0b3e080efa1ad2e476eccf8579d80244bbec4bc318813159d117c8cd88 not found: ID does not exist" containerID="9ec25c0b3e080efa1ad2e476eccf8579d80244bbec4bc318813159d117c8cd88" Jul 22 02:39:31.937759 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:31.937727 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ec25c0b3e080efa1ad2e476eccf8579d80244bbec4bc318813159d117c8cd88"} err="failed to get container status \"9ec25c0b3e080efa1ad2e476eccf8579d80244bbec4bc318813159d117c8cd88\": rpc error: code = NotFound desc = could not find container \"9ec25c0b3e080efa1ad2e476eccf8579d80244bbec4bc318813159d117c8cd88\": container with ID starting with 9ec25c0b3e080efa1ad2e476eccf8579d80244bbec4bc318813159d117c8cd88 not found: ID does not exist" Jul 22 02:39:32.052542 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:32.052500 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fab5c0a7-7259-45e5-b443-dd1887521b2e-extensions-socket-volume\") pod \"fab5c0a7-7259-45e5-b443-dd1887521b2e\" (UID: \"fab5c0a7-7259-45e5-b443-dd1887521b2e\") " Jul 22 02:39:32.052712 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:32.052563 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvjzc\" (UniqueName: \"kubernetes.io/projected/fab5c0a7-7259-45e5-b443-dd1887521b2e-kube-api-access-dvjzc\") pod \"fab5c0a7-7259-45e5-b443-dd1887521b2e\" (UID: \"fab5c0a7-7259-45e5-b443-dd1887521b2e\") " Jul 22 02:39:32.053004 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:32.052979 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fab5c0a7-7259-45e5-b443-dd1887521b2e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fab5c0a7-7259-45e5-b443-dd1887521b2e" (UID: "fab5c0a7-7259-45e5-b443-dd1887521b2e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:39:32.054774 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:32.054754 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fab5c0a7-7259-45e5-b443-dd1887521b2e-kube-api-access-dvjzc" (OuterVolumeSpecName: "kube-api-access-dvjzc") pod "fab5c0a7-7259-45e5-b443-dd1887521b2e" (UID: "fab5c0a7-7259-45e5-b443-dd1887521b2e"). InnerVolumeSpecName "kube-api-access-dvjzc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:39:32.153486 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:32.153452 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvjzc\" (UniqueName: \"kubernetes.io/projected/fab5c0a7-7259-45e5-b443-dd1887521b2e-kube-api-access-dvjzc\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:39:32.153486 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:32.153478 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fab5c0a7-7259-45e5-b443-dd1887521b2e-extensions-socket-volume\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:39:32.250787 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:32.250757 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg"] Jul 22 02:39:32.254877 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:32.254855 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2stxg"] Jul 22 02:39:33.608453 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:39:33.608408 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fab5c0a7-7259-45e5-b443-dd1887521b2e" path="/var/lib/kubelet/pods/fab5c0a7-7259-45e5-b443-dd1887521b2e/volumes" Jul 22 02:40:29.319343 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:29.319308 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tb22g"] Jul 22 02:40:29.319818 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:29.319658 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fab5c0a7-7259-45e5-b443-dd1887521b2e" containerName="manager" Jul 22 02:40:29.319818 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:29.319670 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fab5c0a7-7259-45e5-b443-dd1887521b2e" containerName="manager" Jul 22 02:40:29.319818 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:29.319733 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="fab5c0a7-7259-45e5-b443-dd1887521b2e" containerName="manager" Jul 22 02:40:29.322632 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:29.322615 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tb22g" Jul 22 02:40:29.327129 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:29.327107 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6vhqt\"" Jul 22 02:40:29.334709 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:29.334683 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tb22g"] Jul 22 02:40:29.437049 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:29.437001 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd59t\" (UniqueName: \"kubernetes.io/projected/d2094f78-fc10-4eb6-baea-24169f156c80-kube-api-access-rd59t\") pod \"kuadrant-operator-controller-manager-86878c769f-tb22g\" (UID: \"d2094f78-fc10-4eb6-baea-24169f156c80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tb22g" Jul 22 02:40:29.437221 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:29.437122 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2094f78-fc10-4eb6-baea-24169f156c80-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tb22g\" (UID: \"d2094f78-fc10-4eb6-baea-24169f156c80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tb22g" Jul 22 02:40:29.537535 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:29.537506 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2094f78-fc10-4eb6-baea-24169f156c80-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tb22g\" (UID: \"d2094f78-fc10-4eb6-baea-24169f156c80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tb22g" Jul 22 02:40:29.537703 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:29.537566 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd59t\" (UniqueName: \"kubernetes.io/projected/d2094f78-fc10-4eb6-baea-24169f156c80-kube-api-access-rd59t\") pod \"kuadrant-operator-controller-manager-86878c769f-tb22g\" (UID: \"d2094f78-fc10-4eb6-baea-24169f156c80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tb22g" Jul 22 02:40:29.537896 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:29.537874 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2094f78-fc10-4eb6-baea-24169f156c80-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tb22g\" (UID: \"d2094f78-fc10-4eb6-baea-24169f156c80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tb22g" Jul 22 02:40:29.551133 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:29.551107 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd59t\" (UniqueName: \"kubernetes.io/projected/d2094f78-fc10-4eb6-baea-24169f156c80-kube-api-access-rd59t\") pod \"kuadrant-operator-controller-manager-86878c769f-tb22g\" (UID: \"d2094f78-fc10-4eb6-baea-24169f156c80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tb22g" Jul 22 02:40:29.640344 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:29.640257 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6vhqt\"" Jul 22 02:40:29.643343 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:29.643327 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tb22g" Jul 22 02:40:29.791325 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:29.791296 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tb22g"] Jul 22 02:40:29.794159 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:40:29.794124 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2094f78_fc10_4eb6_baea_24169f156c80.slice/crio-421e80e1c9cecee2620408ac3cb7fb68a0bc6f15a489b8fa13ef5597dc69bc47 WatchSource:0}: Error finding container 421e80e1c9cecee2620408ac3cb7fb68a0bc6f15a489b8fa13ef5597dc69bc47: Status 404 returned error can't find the container with id 421e80e1c9cecee2620408ac3cb7fb68a0bc6f15a489b8fa13ef5597dc69bc47 Jul 22 02:40:30.191008 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:30.190971 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tb22g" event={"ID":"d2094f78-fc10-4eb6-baea-24169f156c80","Type":"ContainerStarted","Data":"180df7badaf3b3361512bf8830ae9b5a139302a32ac4526556bfd8e048cedfec"} Jul 22 02:40:30.191193 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:30.191023 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tb22g" event={"ID":"d2094f78-fc10-4eb6-baea-24169f156c80","Type":"ContainerStarted","Data":"421e80e1c9cecee2620408ac3cb7fb68a0bc6f15a489b8fa13ef5597dc69bc47"} Jul 22 02:40:30.191193 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:30.191079 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tb22g" Jul 22 02:40:30.224016 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:30.223926 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tb22g" podStartSLOduration=1.223905088 podStartE2EDuration="1.223905088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:40:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:40:30.22388396 +0000 UTC m=+1261.132748956" watchObservedRunningTime="2026-07-22 02:40:30.223905088 +0000 UTC m=+1261.132770084" Jul 22 02:40:41.198781 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:40:41.198745 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tb22g" Jul 22 02:47:24.666667 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.666634 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk"] Jul 22 02:47:24.669665 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.669645 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.672344 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.672321 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f7e0def3-openshift-default-dockercfg-rj9k5\"" Jul 22 02:47:24.691674 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.691648 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk"] Jul 22 02:47:24.801953 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.801918 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-credential-socket\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.801953 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.801964 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-envoy\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.802164 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.801995 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-data\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.802164 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.802063 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-workload-certs\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.802164 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.802128 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svv55\" (UniqueName: \"kubernetes.io/projected/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-kube-api-access-svv55\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.802264 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.802182 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istiod-ca-cert\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.802264 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.802224 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-workload-socket\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.802264 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.802247 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-token\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.802393 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.802271 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-podinfo\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.903737 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.903697 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svv55\" (UniqueName: \"kubernetes.io/projected/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-kube-api-access-svv55\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.903950 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.903748 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istiod-ca-cert\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.903950 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.903781 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-workload-socket\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.903950 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.903806 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-token\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.903950 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.903867 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-podinfo\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.903950 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.903902 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-credential-socket\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.903950 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.903922 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-envoy\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.903950 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.903948 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-data\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.904307 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.903994 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-workload-certs\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.904307 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.904275 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-workload-socket\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.904500 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.904469 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-data\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.904631 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.904514 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-workload-certs\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.904631 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.904564 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-credential-socket\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.904631 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.904615 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istiod-ca-cert\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.906510 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.906484 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-envoy\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.906716 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.906696 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-podinfo\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.912750 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.912730 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-token\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.914098 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.914075 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svv55\" (UniqueName: \"kubernetes.io/projected/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-kube-api-access-svv55\") pod \"e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:24.980321 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:24.980292 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:25.121927 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:25.121901 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk"] Jul 22 02:47:25.124235 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:47:25.124205 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e6ac6e3_f1a7_46d4_a3ca_526be4b4048f.slice/crio-580bea8c77d8c603b39f5a0af15b5ed9061538a2b89d774588eb223ba324904b WatchSource:0}: Error finding container 580bea8c77d8c603b39f5a0af15b5ed9061538a2b89d774588eb223ba324904b: Status 404 returned error can't find the container with id 580bea8c77d8c603b39f5a0af15b5ed9061538a2b89d774588eb223ba324904b Jul 22 02:47:25.126814 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:25.126782 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:47:25.126927 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:25.126866 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:47:25.126927 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:25.126893 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:47:25.158251 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:25.158214 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" event={"ID":"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f","Type":"ContainerStarted","Data":"580bea8c77d8c603b39f5a0af15b5ed9061538a2b89d774588eb223ba324904b"} Jul 22 02:47:26.164636 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:26.164601 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" event={"ID":"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f","Type":"ContainerStarted","Data":"5b8fc233e713e759ef24037b13338667f54d7ba508ef1e1fce4309574452bffc"} Jul 22 02:47:26.190042 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:26.189991 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" podStartSLOduration=2.189975991 podStartE2EDuration="2.189975991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:47:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:47:26.189101515 +0000 UTC m=+1677.097966512" watchObservedRunningTime="2026-07-22 02:47:26.189975991 +0000 UTC m=+1677.098840985" Jul 22 02:47:26.981241 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:26.981205 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:26.986319 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:26.986293 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:27.169545 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:27.169503 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:27.170621 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:27.170598 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:41.247690 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:41.247656 2557 generic.go:356] "Generic (PLEG): container finished" podID="3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" containerID="5b8fc233e713e759ef24037b13338667f54d7ba508ef1e1fce4309574452bffc" exitCode=0 Jul 22 02:47:41.248118 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:41.247719 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" event={"ID":"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f","Type":"ContainerDied","Data":"5b8fc233e713e759ef24037b13338667f54d7ba508ef1e1fce4309574452bffc"} Jul 22 02:47:41.248160 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:41.248132 2557 scope.go:117] "RemoveContainer" containerID="5b8fc233e713e759ef24037b13338667f54d7ba508ef1e1fce4309574452bffc" Jul 22 02:47:41.249176 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:41.249144 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:47:41.249250 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:41.249233 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:47:41.249300 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:41.249287 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:47:42.169532 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:42.169479 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:42.254024 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:42.253983 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" event={"ID":"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f","Type":"ContainerStarted","Data":"38413d221117f6aade3ba09863594df6000e89271c834110dafdd77df33af0ea"} Jul 22 02:47:42.981080 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:42.981040 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:42.982138 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:42.982103 2557 patch_prober.go:28] interesting pod/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 22 02:47:42.982285 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:42.982161 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" podUID="3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 22 02:47:43.980645 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:43.980609 2557 patch_prober.go:28] interesting pod/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 22 02:47:43.981166 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:43.980669 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" podUID="3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 22 02:47:44.981014 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:44.980966 2557 patch_prober.go:28] interesting pod/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 22 02:47:44.981401 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:44.981039 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" podUID="3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 22 02:47:45.981397 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:45.981353 2557 patch_prober.go:28] interesting pod/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 22 02:47:45.981885 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:45.981415 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" podUID="3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 22 02:47:46.984904 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:46.984874 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:46.985384 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:46.985250 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:47:46.986016 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:47:46.985995 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:48:08.418996 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:08.418915 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk"] Jul 22 02:48:08.419473 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:08.419171 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" podUID="3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" containerName="istio-proxy" containerID="cri-o://38413d221117f6aade3ba09863594df6000e89271c834110dafdd77df33af0ea" gracePeriod=30 Jul 22 02:48:09.893187 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:09.893153 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9"] Jul 22 02:48:09.896310 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:09.896290 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:09.910643 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:09.910622 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-d0d4ee22-gw-openshift-default-dockercfg-dprgb\"" Jul 22 02:48:09.952631 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:09.952598 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9"] Jul 22 02:48:10.024698 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.024671 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-credential-socket\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.024698 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.024700 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab811e29-71f8-4664-be6a-81f3e4990fa3-istiod-ca-cert\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.024922 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.024735 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-token\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.024922 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.024771 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-envoy\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.024922 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.024868 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-workload-certs\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.024922 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.024893 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-data\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.025089 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.024994 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-podinfo\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.025089 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.025019 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-workload-socket\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.025089 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.025059 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qzgg\" (UniqueName: \"kubernetes.io/projected/ab811e29-71f8-4664-be6a-81f3e4990fa3-kube-api-access-4qzgg\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.126058 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.126019 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-podinfo\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.126253 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.126064 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-workload-socket\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.126253 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.126109 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qzgg\" (UniqueName: \"kubernetes.io/projected/ab811e29-71f8-4664-be6a-81f3e4990fa3-kube-api-access-4qzgg\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.126253 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.126143 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-credential-socket\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.126253 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.126164 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab811e29-71f8-4664-be6a-81f3e4990fa3-istiod-ca-cert\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.126253 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.126219 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-token\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.126253 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.126245 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-envoy\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.126546 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.126280 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-workload-certs\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.126546 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.126482 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-data\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.126713 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.126689 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-workload-socket\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.126807 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.126750 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-workload-certs\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.126879 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.126851 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-credential-socket\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.127076 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.126974 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-data\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.127162 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.127054 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab811e29-71f8-4664-be6a-81f3e4990fa3-istiod-ca-cert\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.128610 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.128586 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-envoy\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.128718 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.128698 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-podinfo\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.145533 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.145475 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-token\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.145656 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.145639 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qzgg\" (UniqueName: \"kubernetes.io/projected/ab811e29-71f8-4664-be6a-81f3e4990fa3-kube-api-access-4qzgg\") pod \"e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.207484 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.207454 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:10.363876 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.363842 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9"] Jul 22 02:48:10.366951 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:48:10.366927 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab811e29_71f8_4664_be6a_81f3e4990fa3.slice/crio-c24a5438ddc379b1eca798f6e2428d68ec3c198eb0b69b32f1a369f5e17a8d7d WatchSource:0}: Error finding container c24a5438ddc379b1eca798f6e2428d68ec3c198eb0b69b32f1a369f5e17a8d7d: Status 404 returned error can't find the container with id c24a5438ddc379b1eca798f6e2428d68ec3c198eb0b69b32f1a369f5e17a8d7d Jul 22 02:48:10.369026 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.368996 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:48:10.369091 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.369060 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:48:10.369127 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.369089 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:48:10.391220 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:10.391191 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" event={"ID":"ab811e29-71f8-4664-be6a-81f3e4990fa3","Type":"ContainerStarted","Data":"c24a5438ddc379b1eca798f6e2428d68ec3c198eb0b69b32f1a369f5e17a8d7d"} Jul 22 02:48:11.397838 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:11.397800 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" event={"ID":"ab811e29-71f8-4664-be6a-81f3e4990fa3","Type":"ContainerStarted","Data":"02b1faf7cf1e0223416002bc617011355c589519c8c14c29c5b29293cffed8a1"} Jul 22 02:48:11.433288 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:11.433240 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" podStartSLOduration=2.433224033 podStartE2EDuration="2.433224033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:48:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:48:11.432383075 +0000 UTC m=+1722.341248070" watchObservedRunningTime="2026-07-22 02:48:11.433224033 +0000 UTC m=+1722.342089027" Jul 22 02:48:12.208132 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:12.208084 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:12.213108 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:12.213081 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:12.402874 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:12.402843 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:12.403691 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:12.403672 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:13.673395 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.673369 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:48:13.759424 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.759398 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-token\") pod \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " Jul 22 02:48:13.759627 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.759434 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-podinfo\") pod \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " Jul 22 02:48:13.759627 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.759474 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svv55\" (UniqueName: \"kubernetes.io/projected/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-kube-api-access-svv55\") pod \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " Jul 22 02:48:13.759627 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.759503 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-workload-socket\") pod \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " Jul 22 02:48:13.759627 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.759551 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-credential-socket\") pod \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " Jul 22 02:48:13.759627 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.759616 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-envoy\") pod \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " Jul 22 02:48:13.759946 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.759657 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-data\") pod \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " Jul 22 02:48:13.759946 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.759693 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-workload-certs\") pod \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " Jul 22 02:48:13.759946 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.759720 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istiod-ca-cert\") pod \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\" (UID: \"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f\") " Jul 22 02:48:13.759946 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.759891 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" (UID: "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:48:13.759946 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.759899 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" (UID: "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:48:13.760293 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.760104 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" (UID: "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:48:13.760293 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.760152 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-credential-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:13.760293 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.760186 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-workload-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:13.760293 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.760197 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" (UID: "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:48:13.760922 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.760893 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-data" (OuterVolumeSpecName: "istio-data") pod "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" (UID: "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:48:13.761942 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.761915 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" (UID: "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:48:13.762067 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.762048 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-token" (OuterVolumeSpecName: "istio-token") pod "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" (UID: "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:48:13.762160 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.762130 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" (UID: "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:48:13.762580 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.762565 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-kube-api-access-svv55" (OuterVolumeSpecName: "kube-api-access-svv55") pod "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" (UID: "3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f"). InnerVolumeSpecName "kube-api-access-svv55". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:48:13.860896 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.860860 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-podinfo\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:13.860896 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.860891 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svv55\" (UniqueName: \"kubernetes.io/projected/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-kube-api-access-svv55\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:13.860896 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.860903 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-envoy\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:13.861129 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.860915 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-data\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:13.861129 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.860925 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-workload-certs\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:13.861129 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.860936 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istiod-ca-cert\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:13.861129 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:13.860946 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f-istio-token\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:14.413670 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:14.413633 2557 generic.go:356] "Generic (PLEG): container finished" podID="3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" containerID="38413d221117f6aade3ba09863594df6000e89271c834110dafdd77df33af0ea" exitCode=0 Jul 22 02:48:14.413851 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:14.413686 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" Jul 22 02:48:14.413851 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:14.413710 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" event={"ID":"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f","Type":"ContainerDied","Data":"38413d221117f6aade3ba09863594df6000e89271c834110dafdd77df33af0ea"} Jul 22 02:48:14.413851 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:14.413767 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk" event={"ID":"3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f","Type":"ContainerDied","Data":"580bea8c77d8c603b39f5a0af15b5ed9061538a2b89d774588eb223ba324904b"} Jul 22 02:48:14.413851 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:14.413789 2557 scope.go:117] "RemoveContainer" containerID="38413d221117f6aade3ba09863594df6000e89271c834110dafdd77df33af0ea" Jul 22 02:48:14.425486 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:14.425418 2557 scope.go:117] "RemoveContainer" containerID="5b8fc233e713e759ef24037b13338667f54d7ba508ef1e1fce4309574452bffc" Jul 22 02:48:14.433345 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:14.433329 2557 scope.go:117] "RemoveContainer" containerID="38413d221117f6aade3ba09863594df6000e89271c834110dafdd77df33af0ea" Jul 22 02:48:14.433596 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:48:14.433579 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38413d221117f6aade3ba09863594df6000e89271c834110dafdd77df33af0ea\": container with ID starting with 38413d221117f6aade3ba09863594df6000e89271c834110dafdd77df33af0ea not found: ID does not exist" containerID="38413d221117f6aade3ba09863594df6000e89271c834110dafdd77df33af0ea" Jul 22 02:48:14.433639 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:14.433606 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38413d221117f6aade3ba09863594df6000e89271c834110dafdd77df33af0ea"} err="failed to get container status \"38413d221117f6aade3ba09863594df6000e89271c834110dafdd77df33af0ea\": rpc error: code = NotFound desc = could not find container \"38413d221117f6aade3ba09863594df6000e89271c834110dafdd77df33af0ea\": container with ID starting with 38413d221117f6aade3ba09863594df6000e89271c834110dafdd77df33af0ea not found: ID does not exist" Jul 22 02:48:14.433639 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:14.433621 2557 scope.go:117] "RemoveContainer" containerID="5b8fc233e713e759ef24037b13338667f54d7ba508ef1e1fce4309574452bffc" Jul 22 02:48:14.433880 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:48:14.433860 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b8fc233e713e759ef24037b13338667f54d7ba508ef1e1fce4309574452bffc\": container with ID starting with 5b8fc233e713e759ef24037b13338667f54d7ba508ef1e1fce4309574452bffc not found: ID does not exist" containerID="5b8fc233e713e759ef24037b13338667f54d7ba508ef1e1fce4309574452bffc" Jul 22 02:48:14.433940 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:14.433887 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b8fc233e713e759ef24037b13338667f54d7ba508ef1e1fce4309574452bffc"} err="failed to get container status \"5b8fc233e713e759ef24037b13338667f54d7ba508ef1e1fce4309574452bffc\": rpc error: code = NotFound desc = could not find container \"5b8fc233e713e759ef24037b13338667f54d7ba508ef1e1fce4309574452bffc\": container with ID starting with 5b8fc233e713e759ef24037b13338667f54d7ba508ef1e1fce4309574452bffc not found: ID does not exist" Jul 22 02:48:14.448078 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:14.448057 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk"] Jul 22 02:48:14.452387 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:14.452364 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f7e0def3-openshift-default-86fcd77d7-dsdpk"] Jul 22 02:48:15.609608 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:15.609569 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" path="/var/lib/kubelet/pods/3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f/volumes" Jul 22 02:48:26.478753 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:26.478715 2557 generic.go:356] "Generic (PLEG): container finished" podID="ab811e29-71f8-4664-be6a-81f3e4990fa3" containerID="02b1faf7cf1e0223416002bc617011355c589519c8c14c29c5b29293cffed8a1" exitCode=0 Jul 22 02:48:26.479178 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:26.478769 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" event={"ID":"ab811e29-71f8-4664-be6a-81f3e4990fa3","Type":"ContainerDied","Data":"02b1faf7cf1e0223416002bc617011355c589519c8c14c29c5b29293cffed8a1"} Jul 22 02:48:26.479232 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:26.479201 2557 scope.go:117] "RemoveContainer" containerID="02b1faf7cf1e0223416002bc617011355c589519c8c14c29c5b29293cffed8a1" Jul 22 02:48:26.480187 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:26.480157 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:48:26.480304 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:26.480241 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:48:26.480304 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:26.480289 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:48:27.403465 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:27.403431 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:27.488436 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:27.488375 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" event={"ID":"ab811e29-71f8-4664-be6a-81f3e4990fa3","Type":"ContainerStarted","Data":"58fb8338cf72e8b1c489250cc95034965b9e89f5c9ba01f82d7710f949925e3a"} Jul 22 02:48:28.208722 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:28.208681 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:28.209940 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:28.209906 2557 patch_prober.go:28] interesting pod/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 22 02:48:28.210080 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:28.209964 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" podUID="ab811e29-71f8-4664-be6a-81f3e4990fa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 22 02:48:29.208742 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:29.208700 2557 patch_prober.go:28] interesting pod/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 22 02:48:29.209160 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:29.208764 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" podUID="ab811e29-71f8-4664-be6a-81f3e4990fa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 22 02:48:30.208539 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:30.208447 2557 patch_prober.go:28] interesting pod/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 22 02:48:30.208539 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:30.208523 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" podUID="ab811e29-71f8-4664-be6a-81f3e4990fa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 22 02:48:31.207907 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:31.207868 2557 patch_prober.go:28] interesting pod/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 22 02:48:31.208378 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:31.207933 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" podUID="ab811e29-71f8-4664-be6a-81f3e4990fa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 22 02:48:32.212445 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:32.212416 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:32.212856 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:32.212667 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:32.213396 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:32.213380 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:53.455425 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:53.455381 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9"] Jul 22 02:48:53.455950 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:53.455612 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" podUID="ab811e29-71f8-4664-be6a-81f3e4990fa3" containerName="istio-proxy" containerID="cri-o://58fb8338cf72e8b1c489250cc95034965b9e89f5c9ba01f82d7710f949925e3a" gracePeriod=30 Jul 22 02:48:54.427159 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.427118 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9"] Jul 22 02:48:54.427495 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.427484 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" containerName="istio-proxy" Jul 22 02:48:54.427539 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.427497 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" containerName="istio-proxy" Jul 22 02:48:54.427576 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.427550 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" containerName="istio-proxy" Jul 22 02:48:54.427576 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.427560 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" containerName="istio-proxy" Jul 22 02:48:54.430585 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.430569 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.434401 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.434370 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-7833c972-openshift-default-dockercfg-8stgc\"" Jul 22 02:48:54.480895 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.480863 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9"] Jul 22 02:48:54.524909 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.524875 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-workload-socket\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.525084 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.524929 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df830f67-bda7-4248-a3dd-42c3a103f701-istio-token\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.525084 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.525005 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-istio-data\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.525084 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.525062 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df830f67-bda7-4248-a3dd-42c3a103f701-istio-podinfo\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.525084 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.525082 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfhkt\" (UniqueName: \"kubernetes.io/projected/df830f67-bda7-4248-a3dd-42c3a103f701-kube-api-access-cfhkt\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.525219 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.525112 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df830f67-bda7-4248-a3dd-42c3a103f701-istiod-ca-cert\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.525219 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.525136 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-istio-envoy\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.525219 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.525161 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-workload-certs\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.525219 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.525189 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-credential-socket\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.626123 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.626090 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-workload-socket\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.626305 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.626131 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df830f67-bda7-4248-a3dd-42c3a103f701-istio-token\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.626305 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.626171 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-istio-data\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.626305 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.626217 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df830f67-bda7-4248-a3dd-42c3a103f701-istio-podinfo\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.626305 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.626242 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfhkt\" (UniqueName: \"kubernetes.io/projected/df830f67-bda7-4248-a3dd-42c3a103f701-kube-api-access-cfhkt\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.626530 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.626421 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df830f67-bda7-4248-a3dd-42c3a103f701-istiod-ca-cert\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.626530 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.626479 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-istio-envoy\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.626530 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.626514 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-workload-certs\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.626676 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.626537 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-workload-socket\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.626676 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.626544 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-credential-socket\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.626676 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.626579 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-istio-data\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.626870 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.626805 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-credential-socket\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.626986 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.626968 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-workload-certs\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.627316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.627294 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df830f67-bda7-4248-a3dd-42c3a103f701-istiod-ca-cert\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.628774 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.628747 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-istio-envoy\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.628984 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.628963 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df830f67-bda7-4248-a3dd-42c3a103f701-istio-podinfo\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.658222 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.658200 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df830f67-bda7-4248-a3dd-42c3a103f701-istio-token\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.658338 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.658279 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfhkt\" (UniqueName: \"kubernetes.io/projected/df830f67-bda7-4248-a3dd-42c3a103f701-kube-api-access-cfhkt\") pod \"e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.745195 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.745166 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:54.905300 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.905259 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9"] Jul 22 02:48:54.916487 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:48:54.916456 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf830f67_bda7_4248_a3dd_42c3a103f701.slice/crio-5e083bfbbaf12d5bce5c40787e4dd35c45d4ce16a285a201981b1e658b8c74c1 WatchSource:0}: Error finding container 5e083bfbbaf12d5bce5c40787e4dd35c45d4ce16a285a201981b1e658b8c74c1: Status 404 returned error can't find the container with id 5e083bfbbaf12d5bce5c40787e4dd35c45d4ce16a285a201981b1e658b8c74c1 Jul 22 02:48:54.918533 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.918504 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:48:54.918624 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.918568 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:48:54.918624 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:54.918597 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:48:55.632861 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:55.632808 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" event={"ID":"df830f67-bda7-4248-a3dd-42c3a103f701","Type":"ContainerStarted","Data":"b3a55e3edfbf40510738f5ebf37284cf4f63cafa3fc9d571132c5bc256af1b0a"} Jul 22 02:48:55.633284 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:55.632873 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" event={"ID":"df830f67-bda7-4248-a3dd-42c3a103f701","Type":"ContainerStarted","Data":"5e083bfbbaf12d5bce5c40787e4dd35c45d4ce16a285a201981b1e658b8c74c1"} Jul 22 02:48:55.672558 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:55.672469 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" podStartSLOduration=1.6724506890000002 podStartE2EDuration="1.672450689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:48:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:48:55.672223706 +0000 UTC m=+1766.581088712" watchObservedRunningTime="2026-07-22 02:48:55.672450689 +0000 UTC m=+1766.581315684" Jul 22 02:48:55.746179 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:55.746150 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:55.750984 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:55.750961 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:56.637183 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:56.637156 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:56.638070 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:56.638051 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:48:58.651185 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.651149 2557 generic.go:356] "Generic (PLEG): container finished" podID="ab811e29-71f8-4664-be6a-81f3e4990fa3" containerID="58fb8338cf72e8b1c489250cc95034965b9e89f5c9ba01f82d7710f949925e3a" exitCode=0 Jul 22 02:48:58.651568 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.651218 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" event={"ID":"ab811e29-71f8-4664-be6a-81f3e4990fa3","Type":"ContainerDied","Data":"58fb8338cf72e8b1c489250cc95034965b9e89f5c9ba01f82d7710f949925e3a"} Jul 22 02:48:58.651568 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.651290 2557 scope.go:117] "RemoveContainer" containerID="02b1faf7cf1e0223416002bc617011355c589519c8c14c29c5b29293cffed8a1" Jul 22 02:48:58.715952 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.715896 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:58.868731 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.868700 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-envoy\") pod \"ab811e29-71f8-4664-be6a-81f3e4990fa3\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " Jul 22 02:48:58.868926 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.868776 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-token\") pod \"ab811e29-71f8-4664-be6a-81f3e4990fa3\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " Jul 22 02:48:58.868926 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.868799 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-workload-certs\") pod \"ab811e29-71f8-4664-be6a-81f3e4990fa3\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " Jul 22 02:48:58.868926 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.868855 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-podinfo\") pod \"ab811e29-71f8-4664-be6a-81f3e4990fa3\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " Jul 22 02:48:58.868926 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.868876 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-workload-socket\") pod \"ab811e29-71f8-4664-be6a-81f3e4990fa3\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " Jul 22 02:48:58.868926 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.868899 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-data\") pod \"ab811e29-71f8-4664-be6a-81f3e4990fa3\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " Jul 22 02:48:58.868926 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.868925 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qzgg\" (UniqueName: \"kubernetes.io/projected/ab811e29-71f8-4664-be6a-81f3e4990fa3-kube-api-access-4qzgg\") pod \"ab811e29-71f8-4664-be6a-81f3e4990fa3\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " Jul 22 02:48:58.869268 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.868963 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-credential-socket\") pod \"ab811e29-71f8-4664-be6a-81f3e4990fa3\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " Jul 22 02:48:58.869268 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.869035 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab811e29-71f8-4664-be6a-81f3e4990fa3-istiod-ca-cert\") pod \"ab811e29-71f8-4664-be6a-81f3e4990fa3\" (UID: \"ab811e29-71f8-4664-be6a-81f3e4990fa3\") " Jul 22 02:48:58.869268 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.869076 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ab811e29-71f8-4664-be6a-81f3e4990fa3" (UID: "ab811e29-71f8-4664-be6a-81f3e4990fa3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:48:58.869425 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.869308 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ab811e29-71f8-4664-be6a-81f3e4990fa3" (UID: "ab811e29-71f8-4664-be6a-81f3e4990fa3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:48:58.869425 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.869326 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-workload-certs\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:58.869712 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.869492 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ab811e29-71f8-4664-be6a-81f3e4990fa3" (UID: "ab811e29-71f8-4664-be6a-81f3e4990fa3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:48:58.869712 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.869560 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab811e29-71f8-4664-be6a-81f3e4990fa3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ab811e29-71f8-4664-be6a-81f3e4990fa3" (UID: "ab811e29-71f8-4664-be6a-81f3e4990fa3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:48:58.869907 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.869803 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-data" (OuterVolumeSpecName: "istio-data") pod "ab811e29-71f8-4664-be6a-81f3e4990fa3" (UID: "ab811e29-71f8-4664-be6a-81f3e4990fa3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:48:58.871099 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.871076 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-token" (OuterVolumeSpecName: "istio-token") pod "ab811e29-71f8-4664-be6a-81f3e4990fa3" (UID: "ab811e29-71f8-4664-be6a-81f3e4990fa3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:48:58.871346 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.871313 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab811e29-71f8-4664-be6a-81f3e4990fa3-kube-api-access-4qzgg" (OuterVolumeSpecName: "kube-api-access-4qzgg") pod "ab811e29-71f8-4664-be6a-81f3e4990fa3" (UID: "ab811e29-71f8-4664-be6a-81f3e4990fa3"). InnerVolumeSpecName "kube-api-access-4qzgg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:48:58.871524 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.871504 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ab811e29-71f8-4664-be6a-81f3e4990fa3" (UID: "ab811e29-71f8-4664-be6a-81f3e4990fa3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:48:58.871591 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.871576 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ab811e29-71f8-4664-be6a-81f3e4990fa3" (UID: "ab811e29-71f8-4664-be6a-81f3e4990fa3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:48:58.970207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.970115 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-podinfo\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:58.970207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.970144 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-workload-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:58.970207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.970157 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-data\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:58.970207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.970170 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qzgg\" (UniqueName: \"kubernetes.io/projected/ab811e29-71f8-4664-be6a-81f3e4990fa3-kube-api-access-4qzgg\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:58.970207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.970181 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-credential-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:58.970207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.970193 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab811e29-71f8-4664-be6a-81f3e4990fa3-istiod-ca-cert\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:58.970207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.970204 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-envoy\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:58.970581 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:58.970215 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab811e29-71f8-4664-be6a-81f3e4990fa3-istio-token\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:48:59.657559 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:59.657525 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" event={"ID":"ab811e29-71f8-4664-be6a-81f3e4990fa3","Type":"ContainerDied","Data":"c24a5438ddc379b1eca798f6e2428d68ec3c198eb0b69b32f1a369f5e17a8d7d"} Jul 22 02:48:59.657559 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:59.657554 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9" Jul 22 02:48:59.658050 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:59.657590 2557 scope.go:117] "RemoveContainer" containerID="58fb8338cf72e8b1c489250cc95034965b9e89f5c9ba01f82d7710f949925e3a" Jul 22 02:48:59.708387 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:59.708344 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9"] Jul 22 02:48:59.710176 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:48:59.710151 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-d0d4ee22-gw-openshift-default-59b6666bc9-q7mv9"] Jul 22 02:49:01.609355 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:01.609322 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab811e29-71f8-4664-be6a-81f3e4990fa3" path="/var/lib/kubelet/pods/ab811e29-71f8-4664-be6a-81f3e4990fa3/volumes" Jul 22 02:49:11.974603 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:11.974565 2557 patch_prober.go:28] interesting pod/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": read tcp 10.133.0.2:34814->10.133.0.44:15021: read: connection reset by peer" start-of-body= Jul 22 02:49:11.975106 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:11.974643 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" podUID="df830f67-bda7-4248-a3dd-42c3a103f701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": read tcp 10.133.0.2:34814->10.133.0.44:15021: read: connection reset by peer" Jul 22 02:49:12.728615 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:12.728582 2557 generic.go:356] "Generic (PLEG): container finished" podID="df830f67-bda7-4248-a3dd-42c3a103f701" containerID="b3a55e3edfbf40510738f5ebf37284cf4f63cafa3fc9d571132c5bc256af1b0a" exitCode=0 Jul 22 02:49:12.728783 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:12.728670 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" event={"ID":"df830f67-bda7-4248-a3dd-42c3a103f701","Type":"ContainerDied","Data":"b3a55e3edfbf40510738f5ebf37284cf4f63cafa3fc9d571132c5bc256af1b0a"} Jul 22 02:49:12.729096 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:12.729080 2557 scope.go:117] "RemoveContainer" containerID="b3a55e3edfbf40510738f5ebf37284cf4f63cafa3fc9d571132c5bc256af1b0a" Jul 22 02:49:12.730132 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:12.730105 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:49:12.730187 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:12.730172 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:49:12.730228 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:12.730215 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:49:13.735002 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:13.734966 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" event={"ID":"df830f67-bda7-4248-a3dd-42c3a103f701","Type":"ContainerStarted","Data":"f524a11a4bf46b2a5c174aa5204620ecd45d952275e3cc964ae9938923967de8"} Jul 22 02:49:13.745469 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:13.745446 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:49:13.745671 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:13.745652 2557 patch_prober.go:28] interesting pod/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 22 02:49:13.745716 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:13.745700 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" podUID="df830f67-bda7-4248-a3dd-42c3a103f701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 22 02:49:14.746349 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:14.746316 2557 patch_prober.go:28] interesting pod/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 22 02:49:14.746766 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:14.746370 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" podUID="df830f67-bda7-4248-a3dd-42c3a103f701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 22 02:49:15.746357 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:15.746322 2557 patch_prober.go:28] interesting pod/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 22 02:49:15.746753 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:15.746383 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" podUID="df830f67-bda7-4248-a3dd-42c3a103f701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 22 02:49:16.745766 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:16.745734 2557 patch_prober.go:28] interesting pod/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 22 02:49:16.745954 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:16.745785 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" podUID="df830f67-bda7-4248-a3dd-42c3a103f701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 22 02:49:17.750058 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:17.750026 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:49:17.750527 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:17.750301 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:49:17.751060 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:17.751039 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:49:39.349098 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:39.349063 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9"] Jul 22 02:49:39.349577 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:39.349314 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" podUID="df830f67-bda7-4248-a3dd-42c3a103f701" containerName="istio-proxy" containerID="cri-o://f524a11a4bf46b2a5c174aa5204620ecd45d952275e3cc964ae9938923967de8" gracePeriod=30 Jul 22 02:49:40.186850 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.186790 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj"] Jul 22 02:49:40.187175 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.187161 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" containerName="istio-proxy" Jul 22 02:49:40.187228 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.187176 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e6ac6e3-f1a7-46d4-a3ca-526be4b4048f" containerName="istio-proxy" Jul 22 02:49:40.187228 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.187200 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab811e29-71f8-4664-be6a-81f3e4990fa3" containerName="istio-proxy" Jul 22 02:49:40.187228 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.187206 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab811e29-71f8-4664-be6a-81f3e4990fa3" containerName="istio-proxy" Jul 22 02:49:40.187228 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.187214 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab811e29-71f8-4664-be6a-81f3e4990fa3" containerName="istio-proxy" Jul 22 02:49:40.187228 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.187220 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab811e29-71f8-4664-be6a-81f3e4990fa3" containerName="istio-proxy" Jul 22 02:49:40.187446 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.187271 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab811e29-71f8-4664-be6a-81f3e4990fa3" containerName="istio-proxy" Jul 22 02:49:40.187446 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.187281 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab811e29-71f8-4664-be6a-81f3e4990fa3" containerName="istio-proxy" Jul 22 02:49:40.190317 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.190298 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.196111 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.196092 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-54774162-openshift-default-dockercfg-wzvhp\"" Jul 22 02:49:40.240157 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.240125 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj"] Jul 22 02:49:40.241109 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.241086 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-workload-certs\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.241230 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.241130 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istiod-ca-cert\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.241230 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.241159 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2knqh\" (UniqueName: \"kubernetes.io/projected/a907f56f-a882-41ee-a02e-8f8ea122a0dc-kube-api-access-2knqh\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.241230 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.241189 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-podinfo\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.241230 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.241219 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-credential-socket\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.241402 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.241277 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-token\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.241402 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.241365 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-data\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.241485 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.241425 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-workload-socket\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.241485 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.241466 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-envoy\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.342716 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.342680 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-data\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.342906 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.342743 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-workload-socket\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.342906 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.342784 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-envoy\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.343031 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.342971 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-workload-certs\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.343087 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.343027 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istiod-ca-cert\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.343087 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.343055 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2knqh\" (UniqueName: \"kubernetes.io/projected/a907f56f-a882-41ee-a02e-8f8ea122a0dc-kube-api-access-2knqh\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.343191 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.343086 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-podinfo\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.343191 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.343119 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-credential-socket\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.343191 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.343161 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-token\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.343191 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.343164 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-data\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.343191 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.343179 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-workload-socket\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.343438 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.343289 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-workload-certs\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.343498 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.343461 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-credential-socket\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.343725 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.343706 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istiod-ca-cert\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.345601 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.345580 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-envoy\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.345699 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.345586 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-podinfo\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.359658 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.359637 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-token\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.368357 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.368333 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2knqh\" (UniqueName: \"kubernetes.io/projected/a907f56f-a882-41ee-a02e-8f8ea122a0dc-kube-api-access-2knqh\") pod \"e2e-derive-54774162-openshift-default-69b766489f-dbkmj\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.502068 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.502035 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:40.651603 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.651579 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj"] Jul 22 02:49:40.653127 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:49:40.653102 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda907f56f_a882_41ee_a02e_8f8ea122a0dc.slice/crio-397400b3414626be2bb75412f1957e90f3d3861daa7c20ce07d651ccf8679504 WatchSource:0}: Error finding container 397400b3414626be2bb75412f1957e90f3d3861daa7c20ce07d651ccf8679504: Status 404 returned error can't find the container with id 397400b3414626be2bb75412f1957e90f3d3861daa7c20ce07d651ccf8679504 Jul 22 02:49:40.655443 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.655411 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:49:40.655515 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.655478 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:49:40.655515 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.655503 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:49:40.871257 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.871149 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" event={"ID":"a907f56f-a882-41ee-a02e-8f8ea122a0dc","Type":"ContainerStarted","Data":"e425f44684842cb55870ec317ebba42289953edf493a9f11d18c8b21055923e2"} Jul 22 02:49:40.871257 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.871204 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" event={"ID":"a907f56f-a882-41ee-a02e-8f8ea122a0dc","Type":"ContainerStarted","Data":"397400b3414626be2bb75412f1957e90f3d3861daa7c20ce07d651ccf8679504"} Jul 22 02:49:40.896021 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:40.895945 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" podStartSLOduration=0.895924095 podStartE2EDuration="895.924095ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:49:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:49:40.893863963 +0000 UTC m=+1811.802728959" watchObservedRunningTime="2026-07-22 02:49:40.895924095 +0000 UTC m=+1811.804789171" Jul 22 02:49:41.502532 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:41.502493 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:41.507418 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:41.507396 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:41.876196 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:41.876122 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:41.877112 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:41.877093 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:44.610253 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.610231 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:49:44.681744 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.681659 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df830f67-bda7-4248-a3dd-42c3a103f701-istio-podinfo\") pod \"df830f67-bda7-4248-a3dd-42c3a103f701\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " Jul 22 02:49:44.681744 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.681703 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-workload-socket\") pod \"df830f67-bda7-4248-a3dd-42c3a103f701\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " Jul 22 02:49:44.681744 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.681722 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df830f67-bda7-4248-a3dd-42c3a103f701-istiod-ca-cert\") pod \"df830f67-bda7-4248-a3dd-42c3a103f701\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " Jul 22 02:49:44.682026 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.681761 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfhkt\" (UniqueName: \"kubernetes.io/projected/df830f67-bda7-4248-a3dd-42c3a103f701-kube-api-access-cfhkt\") pod \"df830f67-bda7-4248-a3dd-42c3a103f701\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " Jul 22 02:49:44.682026 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.681792 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-istio-envoy\") pod \"df830f67-bda7-4248-a3dd-42c3a103f701\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " Jul 22 02:49:44.682026 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.681814 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-istio-data\") pod \"df830f67-bda7-4248-a3dd-42c3a103f701\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " Jul 22 02:49:44.682026 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.681876 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df830f67-bda7-4248-a3dd-42c3a103f701-istio-token\") pod \"df830f67-bda7-4248-a3dd-42c3a103f701\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " Jul 22 02:49:44.682026 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.681930 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-credential-socket\") pod \"df830f67-bda7-4248-a3dd-42c3a103f701\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " Jul 22 02:49:44.682026 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.681991 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-workload-certs\") pod \"df830f67-bda7-4248-a3dd-42c3a103f701\" (UID: \"df830f67-bda7-4248-a3dd-42c3a103f701\") " Jul 22 02:49:44.682327 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.682012 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "df830f67-bda7-4248-a3dd-42c3a103f701" (UID: "df830f67-bda7-4248-a3dd-42c3a103f701"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:49:44.682327 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.682211 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df830f67-bda7-4248-a3dd-42c3a103f701-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "df830f67-bda7-4248-a3dd-42c3a103f701" (UID: "df830f67-bda7-4248-a3dd-42c3a103f701"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:49:44.682327 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.682315 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-workload-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:49:44.682480 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.682334 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df830f67-bda7-4248-a3dd-42c3a103f701-istiod-ca-cert\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:49:44.682480 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.682404 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "df830f67-bda7-4248-a3dd-42c3a103f701" (UID: "df830f67-bda7-4248-a3dd-42c3a103f701"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:49:44.682574 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.682503 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "df830f67-bda7-4248-a3dd-42c3a103f701" (UID: "df830f67-bda7-4248-a3dd-42c3a103f701"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:49:44.682743 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.682721 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-istio-data" (OuterVolumeSpecName: "istio-data") pod "df830f67-bda7-4248-a3dd-42c3a103f701" (UID: "df830f67-bda7-4248-a3dd-42c3a103f701"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:49:44.684290 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.684265 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df830f67-bda7-4248-a3dd-42c3a103f701-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "df830f67-bda7-4248-a3dd-42c3a103f701" (UID: "df830f67-bda7-4248-a3dd-42c3a103f701"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:49:44.684416 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.684397 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df830f67-bda7-4248-a3dd-42c3a103f701-kube-api-access-cfhkt" (OuterVolumeSpecName: "kube-api-access-cfhkt") pod "df830f67-bda7-4248-a3dd-42c3a103f701" (UID: "df830f67-bda7-4248-a3dd-42c3a103f701"). InnerVolumeSpecName "kube-api-access-cfhkt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:49:44.684472 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.684413 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "df830f67-bda7-4248-a3dd-42c3a103f701" (UID: "df830f67-bda7-4248-a3dd-42c3a103f701"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:49:44.684668 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.684648 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df830f67-bda7-4248-a3dd-42c3a103f701-istio-token" (OuterVolumeSpecName: "istio-token") pod "df830f67-bda7-4248-a3dd-42c3a103f701" (UID: "df830f67-bda7-4248-a3dd-42c3a103f701"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:49:44.783324 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.783290 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-credential-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:49:44.783324 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.783322 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-workload-certs\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:49:44.783324 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.783331 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df830f67-bda7-4248-a3dd-42c3a103f701-istio-podinfo\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:49:44.783569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.783340 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfhkt\" (UniqueName: \"kubernetes.io/projected/df830f67-bda7-4248-a3dd-42c3a103f701-kube-api-access-cfhkt\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:49:44.783569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.783350 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-istio-envoy\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:49:44.783569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.783358 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df830f67-bda7-4248-a3dd-42c3a103f701-istio-data\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:49:44.783569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.783365 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df830f67-bda7-4248-a3dd-42c3a103f701-istio-token\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:49:44.892884 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.892843 2557 generic.go:356] "Generic (PLEG): container finished" podID="df830f67-bda7-4248-a3dd-42c3a103f701" containerID="f524a11a4bf46b2a5c174aa5204620ecd45d952275e3cc964ae9938923967de8" exitCode=0 Jul 22 02:49:44.893067 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.892893 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" Jul 22 02:49:44.893067 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.892921 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" event={"ID":"df830f67-bda7-4248-a3dd-42c3a103f701","Type":"ContainerDied","Data":"f524a11a4bf46b2a5c174aa5204620ecd45d952275e3cc964ae9938923967de8"} Jul 22 02:49:44.893067 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.892982 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9" event={"ID":"df830f67-bda7-4248-a3dd-42c3a103f701","Type":"ContainerDied","Data":"5e083bfbbaf12d5bce5c40787e4dd35c45d4ce16a285a201981b1e658b8c74c1"} Jul 22 02:49:44.893067 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.893005 2557 scope.go:117] "RemoveContainer" containerID="f524a11a4bf46b2a5c174aa5204620ecd45d952275e3cc964ae9938923967de8" Jul 22 02:49:44.902537 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.902521 2557 scope.go:117] "RemoveContainer" containerID="b3a55e3edfbf40510738f5ebf37284cf4f63cafa3fc9d571132c5bc256af1b0a" Jul 22 02:49:44.910459 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.910442 2557 scope.go:117] "RemoveContainer" containerID="f524a11a4bf46b2a5c174aa5204620ecd45d952275e3cc964ae9938923967de8" Jul 22 02:49:44.910701 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:49:44.910684 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f524a11a4bf46b2a5c174aa5204620ecd45d952275e3cc964ae9938923967de8\": container with ID starting with f524a11a4bf46b2a5c174aa5204620ecd45d952275e3cc964ae9938923967de8 not found: ID does not exist" containerID="f524a11a4bf46b2a5c174aa5204620ecd45d952275e3cc964ae9938923967de8" Jul 22 02:49:44.910759 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.910708 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f524a11a4bf46b2a5c174aa5204620ecd45d952275e3cc964ae9938923967de8"} err="failed to get container status \"f524a11a4bf46b2a5c174aa5204620ecd45d952275e3cc964ae9938923967de8\": rpc error: code = NotFound desc = could not find container \"f524a11a4bf46b2a5c174aa5204620ecd45d952275e3cc964ae9938923967de8\": container with ID starting with f524a11a4bf46b2a5c174aa5204620ecd45d952275e3cc964ae9938923967de8 not found: ID does not exist" Jul 22 02:49:44.910759 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.910722 2557 scope.go:117] "RemoveContainer" containerID="b3a55e3edfbf40510738f5ebf37284cf4f63cafa3fc9d571132c5bc256af1b0a" Jul 22 02:49:44.911034 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:49:44.910964 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3a55e3edfbf40510738f5ebf37284cf4f63cafa3fc9d571132c5bc256af1b0a\": container with ID starting with b3a55e3edfbf40510738f5ebf37284cf4f63cafa3fc9d571132c5bc256af1b0a not found: ID does not exist" containerID="b3a55e3edfbf40510738f5ebf37284cf4f63cafa3fc9d571132c5bc256af1b0a" Jul 22 02:49:44.911034 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.910985 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3a55e3edfbf40510738f5ebf37284cf4f63cafa3fc9d571132c5bc256af1b0a"} err="failed to get container status \"b3a55e3edfbf40510738f5ebf37284cf4f63cafa3fc9d571132c5bc256af1b0a\": rpc error: code = NotFound desc = could not find container \"b3a55e3edfbf40510738f5ebf37284cf4f63cafa3fc9d571132c5bc256af1b0a\": container with ID starting with b3a55e3edfbf40510738f5ebf37284cf4f63cafa3fc9d571132c5bc256af1b0a not found: ID does not exist" Jul 22 02:49:44.916625 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.916601 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9"] Jul 22 02:49:44.921486 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:44.921465 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-7833c972-openshift-default-55b868bb6c-5kqh9"] Jul 22 02:49:45.608810 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:45.608773 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df830f67-bda7-4248-a3dd-42c3a103f701" path="/var/lib/kubelet/pods/df830f67-bda7-4248-a3dd-42c3a103f701/volumes" Jul 22 02:49:55.950143 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:55.950110 2557 generic.go:356] "Generic (PLEG): container finished" podID="a907f56f-a882-41ee-a02e-8f8ea122a0dc" containerID="e425f44684842cb55870ec317ebba42289953edf493a9f11d18c8b21055923e2" exitCode=0 Jul 22 02:49:55.950618 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:55.950186 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" event={"ID":"a907f56f-a882-41ee-a02e-8f8ea122a0dc","Type":"ContainerDied","Data":"e425f44684842cb55870ec317ebba42289953edf493a9f11d18c8b21055923e2"} Jul 22 02:49:55.950618 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:55.950557 2557 scope.go:117] "RemoveContainer" containerID="e425f44684842cb55870ec317ebba42289953edf493a9f11d18c8b21055923e2" Jul 22 02:49:55.951550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:55.951521 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:49:55.951649 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:55.951606 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:49:55.951710 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:55.951655 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:49:56.876142 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:56.876106 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:56.957040 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:56.956994 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" event={"ID":"a907f56f-a882-41ee-a02e-8f8ea122a0dc","Type":"ContainerStarted","Data":"3a9550495edb746482e4561a7dbf44c98d8ffa40c1f09480ec622f4124a222d6"} Jul 22 02:49:57.504046 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:57.503999 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:49:57.504282 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:57.504149 2557 patch_prober.go:28] interesting pod/e2e-derive-54774162-openshift-default-69b766489f-dbkmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 22 02:49:57.504282 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:57.504199 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" podUID="a907f56f-a882-41ee-a02e-8f8ea122a0dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 22 02:49:58.503282 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:58.503242 2557 patch_prober.go:28] interesting pod/e2e-derive-54774162-openshift-default-69b766489f-dbkmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 22 02:49:58.503925 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:58.503889 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" podUID="a907f56f-a882-41ee-a02e-8f8ea122a0dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 22 02:49:59.502759 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:59.502722 2557 patch_prober.go:28] interesting pod/e2e-derive-54774162-openshift-default-69b766489f-dbkmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 22 02:49:59.503108 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:49:59.503060 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" podUID="a907f56f-a882-41ee-a02e-8f8ea122a0dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 22 02:50:00.503099 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:00.503057 2557 patch_prober.go:28] interesting pod/e2e-derive-54774162-openshift-default-69b766489f-dbkmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 22 02:50:00.503851 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:00.503781 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" podUID="a907f56f-a882-41ee-a02e-8f8ea122a0dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 22 02:50:01.506768 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:01.506737 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:50:01.507142 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:01.507030 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:50:01.507710 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:01.507692 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:50:22.411073 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:22.411025 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj"] Jul 22 02:50:22.411586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:22.411309 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" podUID="a907f56f-a882-41ee-a02e-8f8ea122a0dc" containerName="istio-proxy" containerID="cri-o://3a9550495edb746482e4561a7dbf44c98d8ffa40c1f09480ec622f4124a222d6" gracePeriod=30 Jul 22 02:50:27.666105 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.666071 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:50:27.770362 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.770325 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-data\") pod \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " Jul 22 02:50:27.770570 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.770389 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2knqh\" (UniqueName: \"kubernetes.io/projected/a907f56f-a882-41ee-a02e-8f8ea122a0dc-kube-api-access-2knqh\") pod \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " Jul 22 02:50:27.770570 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.770432 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-workload-certs\") pod \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " Jul 22 02:50:27.770570 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.770447 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-credential-socket\") pod \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " Jul 22 02:50:27.770570 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.770478 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-envoy\") pod \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " Jul 22 02:50:27.770570 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.770510 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-token\") pod \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " Jul 22 02:50:27.770570 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.770544 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-workload-socket\") pod \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " Jul 22 02:50:27.770906 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.770584 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istiod-ca-cert\") pod \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " Jul 22 02:50:27.770906 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.770611 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-podinfo\") pod \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\" (UID: \"a907f56f-a882-41ee-a02e-8f8ea122a0dc\") " Jul 22 02:50:27.770906 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.770673 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a907f56f-a882-41ee-a02e-8f8ea122a0dc" (UID: "a907f56f-a882-41ee-a02e-8f8ea122a0dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:50:27.770906 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.770697 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a907f56f-a882-41ee-a02e-8f8ea122a0dc" (UID: "a907f56f-a882-41ee-a02e-8f8ea122a0dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:50:27.770906 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.770869 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a907f56f-a882-41ee-a02e-8f8ea122a0dc" (UID: "a907f56f-a882-41ee-a02e-8f8ea122a0dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:50:27.771196 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.770961 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-workload-certs\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:50:27.771196 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.770976 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-credential-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:50:27.771196 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.770989 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-workload-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:50:27.771196 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.771113 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a907f56f-a882-41ee-a02e-8f8ea122a0dc" (UID: "a907f56f-a882-41ee-a02e-8f8ea122a0dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:50:27.771196 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.771151 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-data" (OuterVolumeSpecName: "istio-data") pod "a907f56f-a882-41ee-a02e-8f8ea122a0dc" (UID: "a907f56f-a882-41ee-a02e-8f8ea122a0dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:50:27.772948 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.772919 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a907f56f-a882-41ee-a02e-8f8ea122a0dc-kube-api-access-2knqh" (OuterVolumeSpecName: "kube-api-access-2knqh") pod "a907f56f-a882-41ee-a02e-8f8ea122a0dc" (UID: "a907f56f-a882-41ee-a02e-8f8ea122a0dc"). InnerVolumeSpecName "kube-api-access-2knqh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:50:27.772948 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.772931 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-token" (OuterVolumeSpecName: "istio-token") pod "a907f56f-a882-41ee-a02e-8f8ea122a0dc" (UID: "a907f56f-a882-41ee-a02e-8f8ea122a0dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:50:27.773255 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.773239 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a907f56f-a882-41ee-a02e-8f8ea122a0dc" (UID: "a907f56f-a882-41ee-a02e-8f8ea122a0dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:50:27.773374 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.773356 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a907f56f-a882-41ee-a02e-8f8ea122a0dc" (UID: "a907f56f-a882-41ee-a02e-8f8ea122a0dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:50:27.871437 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.871399 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-data\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:50:27.871437 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.871428 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2knqh\" (UniqueName: \"kubernetes.io/projected/a907f56f-a882-41ee-a02e-8f8ea122a0dc-kube-api-access-2knqh\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:50:27.871437 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.871438 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-envoy\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:50:27.871437 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.871446 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-token\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:50:27.871437 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.871454 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istiod-ca-cert\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:50:27.871959 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:27.871461 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a907f56f-a882-41ee-a02e-8f8ea122a0dc-istio-podinfo\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:50:28.114299 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:28.114197 2557 generic.go:356] "Generic (PLEG): container finished" podID="a907f56f-a882-41ee-a02e-8f8ea122a0dc" containerID="3a9550495edb746482e4561a7dbf44c98d8ffa40c1f09480ec622f4124a222d6" exitCode=0 Jul 22 02:50:28.114299 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:28.114245 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" event={"ID":"a907f56f-a882-41ee-a02e-8f8ea122a0dc","Type":"ContainerDied","Data":"3a9550495edb746482e4561a7dbf44c98d8ffa40c1f09480ec622f4124a222d6"} Jul 22 02:50:28.114299 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:28.114280 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" Jul 22 02:50:28.114577 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:28.114331 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj" event={"ID":"a907f56f-a882-41ee-a02e-8f8ea122a0dc","Type":"ContainerDied","Data":"397400b3414626be2bb75412f1957e90f3d3861daa7c20ce07d651ccf8679504"} Jul 22 02:50:28.114577 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:28.114361 2557 scope.go:117] "RemoveContainer" containerID="3a9550495edb746482e4561a7dbf44c98d8ffa40c1f09480ec622f4124a222d6" Jul 22 02:50:28.127337 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:28.127244 2557 scope.go:117] "RemoveContainer" containerID="e425f44684842cb55870ec317ebba42289953edf493a9f11d18c8b21055923e2" Jul 22 02:50:28.136037 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:28.136018 2557 scope.go:117] "RemoveContainer" containerID="3a9550495edb746482e4561a7dbf44c98d8ffa40c1f09480ec622f4124a222d6" Jul 22 02:50:28.136332 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:50:28.136311 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a9550495edb746482e4561a7dbf44c98d8ffa40c1f09480ec622f4124a222d6\": container with ID starting with 3a9550495edb746482e4561a7dbf44c98d8ffa40c1f09480ec622f4124a222d6 not found: ID does not exist" containerID="3a9550495edb746482e4561a7dbf44c98d8ffa40c1f09480ec622f4124a222d6" Jul 22 02:50:28.136383 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:28.136342 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a9550495edb746482e4561a7dbf44c98d8ffa40c1f09480ec622f4124a222d6"} err="failed to get container status \"3a9550495edb746482e4561a7dbf44c98d8ffa40c1f09480ec622f4124a222d6\": rpc error: code = NotFound desc = could not find container \"3a9550495edb746482e4561a7dbf44c98d8ffa40c1f09480ec622f4124a222d6\": container with ID starting with 3a9550495edb746482e4561a7dbf44c98d8ffa40c1f09480ec622f4124a222d6 not found: ID does not exist" Jul 22 02:50:28.136383 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:28.136360 2557 scope.go:117] "RemoveContainer" containerID="e425f44684842cb55870ec317ebba42289953edf493a9f11d18c8b21055923e2" Jul 22 02:50:28.136611 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:50:28.136597 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e425f44684842cb55870ec317ebba42289953edf493a9f11d18c8b21055923e2\": container with ID starting with e425f44684842cb55870ec317ebba42289953edf493a9f11d18c8b21055923e2 not found: ID does not exist" containerID="e425f44684842cb55870ec317ebba42289953edf493a9f11d18c8b21055923e2" Jul 22 02:50:28.136648 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:28.136615 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e425f44684842cb55870ec317ebba42289953edf493a9f11d18c8b21055923e2"} err="failed to get container status \"e425f44684842cb55870ec317ebba42289953edf493a9f11d18c8b21055923e2\": rpc error: code = NotFound desc = could not find container \"e425f44684842cb55870ec317ebba42289953edf493a9f11d18c8b21055923e2\": container with ID starting with e425f44684842cb55870ec317ebba42289953edf493a9f11d18c8b21055923e2 not found: ID does not exist" Jul 22 02:50:28.142835 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:28.142796 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj"] Jul 22 02:50:28.148997 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:28.148975 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-54774162-openshift-default-69b766489f-dbkmj"] Jul 22 02:50:29.609936 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:50:29.609905 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a907f56f-a882-41ee-a02e-8f8ea122a0dc" path="/var/lib/kubelet/pods/a907f56f-a882-41ee-a02e-8f8ea122a0dc/volumes" Jul 22 02:52:03.281695 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.281658 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9"] Jul 22 02:52:03.282302 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.282036 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a907f56f-a882-41ee-a02e-8f8ea122a0dc" containerName="istio-proxy" Jul 22 02:52:03.282302 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.282048 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a907f56f-a882-41ee-a02e-8f8ea122a0dc" containerName="istio-proxy" Jul 22 02:52:03.282302 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.282070 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df830f67-bda7-4248-a3dd-42c3a103f701" containerName="istio-proxy" Jul 22 02:52:03.282302 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.282077 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="df830f67-bda7-4248-a3dd-42c3a103f701" containerName="istio-proxy" Jul 22 02:52:03.282302 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.282091 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df830f67-bda7-4248-a3dd-42c3a103f701" containerName="istio-proxy" Jul 22 02:52:03.282302 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.282096 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="df830f67-bda7-4248-a3dd-42c3a103f701" containerName="istio-proxy" Jul 22 02:52:03.282302 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.282157 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="df830f67-bda7-4248-a3dd-42c3a103f701" containerName="istio-proxy" Jul 22 02:52:03.282302 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.282167 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a907f56f-a882-41ee-a02e-8f8ea122a0dc" containerName="istio-proxy" Jul 22 02:52:03.282302 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.282174 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a907f56f-a882-41ee-a02e-8f8ea122a0dc" containerName="istio-proxy" Jul 22 02:52:03.285197 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.285175 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.288163 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.288142 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4151b7b4-openshift-default-dockercfg-6fx9b\"" Jul 22 02:52:03.307002 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.306973 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9"] Jul 22 02:52:03.410687 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.410645 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-data\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.410919 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.410744 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-workload-socket\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.410919 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.410787 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-token\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.410919 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.410843 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-workload-certs\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.410919 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.410890 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-podinfo\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.411089 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.410986 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-envoy\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.411089 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.411020 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-credential-socket\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.411089 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.411052 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istiod-ca-cert\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.411089 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.411079 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85h2g\" (UniqueName: \"kubernetes.io/projected/7ce54b3f-5cd8-4116-af2e-6fd175a69807-kube-api-access-85h2g\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.511770 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.511728 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istiod-ca-cert\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.511770 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.511767 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85h2g\" (UniqueName: \"kubernetes.io/projected/7ce54b3f-5cd8-4116-af2e-6fd175a69807-kube-api-access-85h2g\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.512037 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.511793 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-data\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.512037 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.511848 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-workload-socket\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.512037 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.511866 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-token\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.512037 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.511891 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-workload-certs\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.512037 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.511909 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-podinfo\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.512037 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.511969 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-envoy\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.512037 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.512008 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-credential-socket\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.512357 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.512314 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-workload-socket\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.512422 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.512364 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-workload-certs\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.512422 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.512394 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-credential-socket\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.512519 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.512440 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-data\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.512629 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.512604 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istiod-ca-cert\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.514550 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.514529 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-envoy\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.514801 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.514780 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-podinfo\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.522788 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.522761 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-token\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.523149 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.523129 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85h2g\" (UniqueName: \"kubernetes.io/projected/7ce54b3f-5cd8-4116-af2e-6fd175a69807-kube-api-access-85h2g\") pod \"e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.597428 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.597343 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:03.744519 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.744491 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9"] Jul 22 02:52:03.747271 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:52:03.747246 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce54b3f_5cd8_4116_af2e_6fd175a69807.slice/crio-fb3707c9b7db5dd6f2b234aa0640d38caa116e74cadaab918a36291fd754ed51 WatchSource:0}: Error finding container fb3707c9b7db5dd6f2b234aa0640d38caa116e74cadaab918a36291fd754ed51: Status 404 returned error can't find the container with id fb3707c9b7db5dd6f2b234aa0640d38caa116e74cadaab918a36291fd754ed51 Jul 22 02:52:03.749387 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.749357 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:52:03.749461 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.749419 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:52:03.749461 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:03.749454 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:52:04.579815 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:04.579781 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" event={"ID":"7ce54b3f-5cd8-4116-af2e-6fd175a69807","Type":"ContainerStarted","Data":"59da63f6a96a11294390de0e1ec14187cbb9dea44ec767c7008a1b6f26a5d673"} Jul 22 02:52:04.580205 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:04.579862 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" event={"ID":"7ce54b3f-5cd8-4116-af2e-6fd175a69807","Type":"ContainerStarted","Data":"fb3707c9b7db5dd6f2b234aa0640d38caa116e74cadaab918a36291fd754ed51"} Jul 22 02:52:04.597839 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:04.597791 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:04.603158 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:04.603127 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:04.607621 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:04.607578 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" podStartSLOduration=1.6075620750000001 podStartE2EDuration="1.607562075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:52:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:52:04.605250916 +0000 UTC m=+1955.514115910" watchObservedRunningTime="2026-07-22 02:52:04.607562075 +0000 UTC m=+1955.516427069" Jul 22 02:52:05.585615 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:05.585581 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:05.586967 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:05.586942 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:19.656276 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:19.656240 2557 generic.go:356] "Generic (PLEG): container finished" podID="7ce54b3f-5cd8-4116-af2e-6fd175a69807" containerID="59da63f6a96a11294390de0e1ec14187cbb9dea44ec767c7008a1b6f26a5d673" exitCode=0 Jul 22 02:52:19.656690 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:19.656315 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" event={"ID":"7ce54b3f-5cd8-4116-af2e-6fd175a69807","Type":"ContainerDied","Data":"59da63f6a96a11294390de0e1ec14187cbb9dea44ec767c7008a1b6f26a5d673"} Jul 22 02:52:19.656732 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:19.656709 2557 scope.go:117] "RemoveContainer" containerID="59da63f6a96a11294390de0e1ec14187cbb9dea44ec767c7008a1b6f26a5d673" Jul 22 02:52:19.657784 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:19.657759 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:52:19.657884 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:19.657867 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:52:19.657929 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:19.657914 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:52:20.585636 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:20.585599 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:20.663331 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:20.663296 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" event={"ID":"7ce54b3f-5cd8-4116-af2e-6fd175a69807","Type":"ContainerStarted","Data":"95916d358b8bbe6b47e6a254c07e5a87d0d628a3a5742aff7517eb46ac2ae0d5"} Jul 22 02:52:21.598291 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:21.598240 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:21.599475 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:21.599434 2557 patch_prober.go:28] interesting pod/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 22 02:52:21.599726 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:21.599671 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" podUID="7ce54b3f-5cd8-4116-af2e-6fd175a69807" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 22 02:52:22.598871 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:22.598799 2557 patch_prober.go:28] interesting pod/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 22 02:52:22.599380 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:22.598913 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" podUID="7ce54b3f-5cd8-4116-af2e-6fd175a69807" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 22 02:52:23.598284 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:23.598244 2557 patch_prober.go:28] interesting pod/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 22 02:52:23.598584 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:23.598556 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" podUID="7ce54b3f-5cd8-4116-af2e-6fd175a69807" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 22 02:52:24.601817 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:24.601782 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:24.602201 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:24.602090 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:24.602839 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:24.602801 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:39.805716 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:39.805682 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9"] Jul 22 02:52:39.806257 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:39.805959 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" podUID="7ce54b3f-5cd8-4116-af2e-6fd175a69807" containerName="istio-proxy" containerID="cri-o://95916d358b8bbe6b47e6a254c07e5a87d0d628a3a5742aff7517eb46ac2ae0d5" gracePeriod=30 Jul 22 02:52:45.065202 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.065171 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:45.109762 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.109679 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-workload-socket\") pod \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " Jul 22 02:52:45.109762 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.109725 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-token\") pod \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " Jul 22 02:52:45.110008 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.109771 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-envoy\") pod \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " Jul 22 02:52:45.110008 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.109805 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-workload-certs\") pod \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " Jul 22 02:52:45.110008 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.109867 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-data\") pod \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " Jul 22 02:52:45.110008 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.109945 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-podinfo\") pod \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " Jul 22 02:52:45.110008 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.109970 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-credential-socket\") pod \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " Jul 22 02:52:45.110275 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.110009 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7ce54b3f-5cd8-4116-af2e-6fd175a69807" (UID: "7ce54b3f-5cd8-4116-af2e-6fd175a69807"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:52:45.110275 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.110042 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85h2g\" (UniqueName: \"kubernetes.io/projected/7ce54b3f-5cd8-4116-af2e-6fd175a69807-kube-api-access-85h2g\") pod \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " Jul 22 02:52:45.110275 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.110072 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istiod-ca-cert\") pod \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\" (UID: \"7ce54b3f-5cd8-4116-af2e-6fd175a69807\") " Jul 22 02:52:45.110275 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.110259 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7ce54b3f-5cd8-4116-af2e-6fd175a69807" (UID: "7ce54b3f-5cd8-4116-af2e-6fd175a69807"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:52:45.110471 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.110384 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-credential-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:52:45.110471 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.110382 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7ce54b3f-5cd8-4116-af2e-6fd175a69807" (UID: "7ce54b3f-5cd8-4116-af2e-6fd175a69807"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:52:45.110471 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.110403 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-workload-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:52:45.110600 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.110578 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7ce54b3f-5cd8-4116-af2e-6fd175a69807" (UID: "7ce54b3f-5cd8-4116-af2e-6fd175a69807"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:52:45.111039 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.111007 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-data" (OuterVolumeSpecName: "istio-data") pod "7ce54b3f-5cd8-4116-af2e-6fd175a69807" (UID: "7ce54b3f-5cd8-4116-af2e-6fd175a69807"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:52:45.112365 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.112331 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7ce54b3f-5cd8-4116-af2e-6fd175a69807" (UID: "7ce54b3f-5cd8-4116-af2e-6fd175a69807"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:52:45.112489 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.112410 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-token" (OuterVolumeSpecName: "istio-token") pod "7ce54b3f-5cd8-4116-af2e-6fd175a69807" (UID: "7ce54b3f-5cd8-4116-af2e-6fd175a69807"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:52:45.112916 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.112898 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7ce54b3f-5cd8-4116-af2e-6fd175a69807" (UID: "7ce54b3f-5cd8-4116-af2e-6fd175a69807"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:52:45.113022 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.113005 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce54b3f-5cd8-4116-af2e-6fd175a69807-kube-api-access-85h2g" (OuterVolumeSpecName: "kube-api-access-85h2g") pod "7ce54b3f-5cd8-4116-af2e-6fd175a69807" (UID: "7ce54b3f-5cd8-4116-af2e-6fd175a69807"). InnerVolumeSpecName "kube-api-access-85h2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:52:45.211253 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.211212 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85h2g\" (UniqueName: \"kubernetes.io/projected/7ce54b3f-5cd8-4116-af2e-6fd175a69807-kube-api-access-85h2g\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:52:45.211253 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.211246 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istiod-ca-cert\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:52:45.211253 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.211258 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-token\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:52:45.211496 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.211270 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-envoy\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:52:45.211496 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.211284 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-workload-certs\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:52:45.211496 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.211294 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-data\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:52:45.211496 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.211304 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce54b3f-5cd8-4116-af2e-6fd175a69807-istio-podinfo\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:52:45.782060 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.782026 2557 generic.go:356] "Generic (PLEG): container finished" podID="7ce54b3f-5cd8-4116-af2e-6fd175a69807" containerID="95916d358b8bbe6b47e6a254c07e5a87d0d628a3a5742aff7517eb46ac2ae0d5" exitCode=0 Jul 22 02:52:45.782241 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.782060 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" event={"ID":"7ce54b3f-5cd8-4116-af2e-6fd175a69807","Type":"ContainerDied","Data":"95916d358b8bbe6b47e6a254c07e5a87d0d628a3a5742aff7517eb46ac2ae0d5"} Jul 22 02:52:45.782241 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.782082 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" Jul 22 02:52:45.782241 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.782114 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9" event={"ID":"7ce54b3f-5cd8-4116-af2e-6fd175a69807","Type":"ContainerDied","Data":"fb3707c9b7db5dd6f2b234aa0640d38caa116e74cadaab918a36291fd754ed51"} Jul 22 02:52:45.782241 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.782138 2557 scope.go:117] "RemoveContainer" containerID="95916d358b8bbe6b47e6a254c07e5a87d0d628a3a5742aff7517eb46ac2ae0d5" Jul 22 02:52:45.791085 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.791063 2557 scope.go:117] "RemoveContainer" containerID="59da63f6a96a11294390de0e1ec14187cbb9dea44ec767c7008a1b6f26a5d673" Jul 22 02:52:45.798753 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.798736 2557 scope.go:117] "RemoveContainer" containerID="95916d358b8bbe6b47e6a254c07e5a87d0d628a3a5742aff7517eb46ac2ae0d5" Jul 22 02:52:45.799044 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:52:45.799022 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95916d358b8bbe6b47e6a254c07e5a87d0d628a3a5742aff7517eb46ac2ae0d5\": container with ID starting with 95916d358b8bbe6b47e6a254c07e5a87d0d628a3a5742aff7517eb46ac2ae0d5 not found: ID does not exist" containerID="95916d358b8bbe6b47e6a254c07e5a87d0d628a3a5742aff7517eb46ac2ae0d5" Jul 22 02:52:45.799114 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.799054 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95916d358b8bbe6b47e6a254c07e5a87d0d628a3a5742aff7517eb46ac2ae0d5"} err="failed to get container status \"95916d358b8bbe6b47e6a254c07e5a87d0d628a3a5742aff7517eb46ac2ae0d5\": rpc error: code = NotFound desc = could not find container \"95916d358b8bbe6b47e6a254c07e5a87d0d628a3a5742aff7517eb46ac2ae0d5\": container with ID starting with 95916d358b8bbe6b47e6a254c07e5a87d0d628a3a5742aff7517eb46ac2ae0d5 not found: ID does not exist" Jul 22 02:52:45.799114 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.799072 2557 scope.go:117] "RemoveContainer" containerID="59da63f6a96a11294390de0e1ec14187cbb9dea44ec767c7008a1b6f26a5d673" Jul 22 02:52:45.799292 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:52:45.799274 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59da63f6a96a11294390de0e1ec14187cbb9dea44ec767c7008a1b6f26a5d673\": container with ID starting with 59da63f6a96a11294390de0e1ec14187cbb9dea44ec767c7008a1b6f26a5d673 not found: ID does not exist" containerID="59da63f6a96a11294390de0e1ec14187cbb9dea44ec767c7008a1b6f26a5d673" Jul 22 02:52:45.799336 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.799298 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59da63f6a96a11294390de0e1ec14187cbb9dea44ec767c7008a1b6f26a5d673"} err="failed to get container status \"59da63f6a96a11294390de0e1ec14187cbb9dea44ec767c7008a1b6f26a5d673\": rpc error: code = NotFound desc = could not find container \"59da63f6a96a11294390de0e1ec14187cbb9dea44ec767c7008a1b6f26a5d673\": container with ID starting with 59da63f6a96a11294390de0e1ec14187cbb9dea44ec767c7008a1b6f26a5d673 not found: ID does not exist" Jul 22 02:52:45.809001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.808980 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9"] Jul 22 02:52:45.816634 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:45.816612 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4151b7b4-openshift-default-654fb7659-rtjf9"] Jul 22 02:52:47.609104 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:52:47.609067 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ce54b3f-5cd8-4116-af2e-6fd175a69807" path="/var/lib/kubelet/pods/7ce54b3f-5cd8-4116-af2e-6fd175a69807/volumes" Jul 22 02:53:00.694103 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.694068 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt"] Jul 22 02:53:00.694563 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.694413 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ce54b3f-5cd8-4116-af2e-6fd175a69807" containerName="istio-proxy" Jul 22 02:53:00.694563 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.694424 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ce54b3f-5cd8-4116-af2e-6fd175a69807" containerName="istio-proxy" Jul 22 02:53:00.694563 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.694437 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a907f56f-a882-41ee-a02e-8f8ea122a0dc" containerName="istio-proxy" Jul 22 02:53:00.694563 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.694443 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a907f56f-a882-41ee-a02e-8f8ea122a0dc" containerName="istio-proxy" Jul 22 02:53:00.694563 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.694450 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ce54b3f-5cd8-4116-af2e-6fd175a69807" containerName="istio-proxy" Jul 22 02:53:00.694563 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.694455 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ce54b3f-5cd8-4116-af2e-6fd175a69807" containerName="istio-proxy" Jul 22 02:53:00.694563 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.694509 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="df830f67-bda7-4248-a3dd-42c3a103f701" containerName="istio-proxy" Jul 22 02:53:00.694563 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.694518 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ce54b3f-5cd8-4116-af2e-6fd175a69807" containerName="istio-proxy" Jul 22 02:53:00.698871 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.698850 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.702924 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.702894 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6a145fe4-openshift-default-dockercfg-nj6nc\"" Jul 22 02:53:00.796208 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.796172 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt"] Jul 22 02:53:00.852134 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.852102 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-credential-socket\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.852299 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.852146 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-workload-certs\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.852299 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.852212 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-data\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.852299 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.852279 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-envoy\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.852432 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.852310 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpl7k\" (UniqueName: \"kubernetes.io/projected/75e2b709-904d-45a7-ac95-a3ae526e1c43-kube-api-access-bpl7k\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.852470 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.852428 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75e2b709-904d-45a7-ac95-a3ae526e1c43-istiod-ca-cert\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.852522 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.852478 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-podinfo\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.852585 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.852530 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-workload-socket\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.852585 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.852565 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-token\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.954077 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.953993 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-data\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.954077 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.954035 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-envoy\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.954077 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.954058 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpl7k\" (UniqueName: \"kubernetes.io/projected/75e2b709-904d-45a7-ac95-a3ae526e1c43-kube-api-access-bpl7k\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.954357 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.954113 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75e2b709-904d-45a7-ac95-a3ae526e1c43-istiod-ca-cert\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.954357 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.954157 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-podinfo\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.954357 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.954276 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-workload-socket\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.954357 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.954319 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-token\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.954569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.954384 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-credential-socket\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.954569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.954414 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-data\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.954569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.954428 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-workload-certs\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.954738 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.954599 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-workload-socket\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.954738 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.954687 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-workload-certs\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.954738 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.954708 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-credential-socket\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.954948 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.954901 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75e2b709-904d-45a7-ac95-a3ae526e1c43-istiod-ca-cert\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.957075 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.957036 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-envoy\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.957183 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.957160 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-podinfo\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.963122 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.963100 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-token\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:00.963235 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:00.963218 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpl7k\" (UniqueName: \"kubernetes.io/projected/75e2b709-904d-45a7-ac95-a3ae526e1c43-kube-api-access-bpl7k\") pod \"e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:01.009036 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:01.009005 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:01.157791 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:01.157751 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt"] Jul 22 02:53:01.161953 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:53:01.161930 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75e2b709_904d_45a7_ac95_a3ae526e1c43.slice/crio-5eeb43bda3ab34729cb027126de1a45ca4cc016a0b312b13c27b1a44308e5305 WatchSource:0}: Error finding container 5eeb43bda3ab34729cb027126de1a45ca4cc016a0b312b13c27b1a44308e5305: Status 404 returned error can't find the container with id 5eeb43bda3ab34729cb027126de1a45ca4cc016a0b312b13c27b1a44308e5305 Jul 22 02:53:01.164367 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:01.164332 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:53:01.164459 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:01.164424 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:53:01.164459 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:01.164454 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:53:01.862885 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:01.862852 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" event={"ID":"75e2b709-904d-45a7-ac95-a3ae526e1c43","Type":"ContainerStarted","Data":"52e099b837f55150b16ac56a155a61577ca10f43f43765af14ceed1b03b9127b"} Jul 22 02:53:01.863253 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:01.862902 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" event={"ID":"75e2b709-904d-45a7-ac95-a3ae526e1c43","Type":"ContainerStarted","Data":"5eeb43bda3ab34729cb027126de1a45ca4cc016a0b312b13c27b1a44308e5305"} Jul 22 02:53:01.889205 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:01.889156 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" podStartSLOduration=1.889140571 podStartE2EDuration="1.889140571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:53:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:53:01.887069998 +0000 UTC m=+2012.795934994" watchObservedRunningTime="2026-07-22 02:53:01.889140571 +0000 UTC m=+2012.798005565" Jul 22 02:53:02.009450 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:02.009411 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:02.014263 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:02.014237 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:02.868668 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:02.868635 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:02.869721 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:02.869697 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:16.938319 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:16.938280 2557 generic.go:356] "Generic (PLEG): container finished" podID="75e2b709-904d-45a7-ac95-a3ae526e1c43" containerID="52e099b837f55150b16ac56a155a61577ca10f43f43765af14ceed1b03b9127b" exitCode=0 Jul 22 02:53:16.938785 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:16.938331 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" event={"ID":"75e2b709-904d-45a7-ac95-a3ae526e1c43","Type":"ContainerDied","Data":"52e099b837f55150b16ac56a155a61577ca10f43f43765af14ceed1b03b9127b"} Jul 22 02:53:16.938785 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:16.938700 2557 scope.go:117] "RemoveContainer" containerID="52e099b837f55150b16ac56a155a61577ca10f43f43765af14ceed1b03b9127b" Jul 22 02:53:16.939653 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:16.939624 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:53:16.939746 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:16.939691 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:53:16.939746 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:16.939722 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:53:17.868942 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:17.868904 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:17.945341 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:17.945305 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" event={"ID":"75e2b709-904d-45a7-ac95-a3ae526e1c43","Type":"ContainerStarted","Data":"f3fc5eaa8e6d028478e35b1cd1b2b76de124bf085464165dee42c831b5cbd61b"} Jul 22 02:53:18.009572 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:18.009521 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:18.010789 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:18.010759 2557 patch_prober.go:28] interesting pod/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 22 02:53:18.010941 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:18.010816 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" podUID="75e2b709-904d-45a7-ac95-a3ae526e1c43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 22 02:53:19.010877 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:19.010368 2557 patch_prober.go:28] interesting pod/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 22 02:53:19.010877 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:19.010441 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" podUID="75e2b709-904d-45a7-ac95-a3ae526e1c43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 22 02:53:20.010015 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:20.009973 2557 patch_prober.go:28] interesting pod/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 22 02:53:20.010207 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:20.010043 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" podUID="75e2b709-904d-45a7-ac95-a3ae526e1c43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 22 02:53:21.010526 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:21.010486 2557 patch_prober.go:28] interesting pod/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 22 02:53:21.010944 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:21.010556 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" podUID="75e2b709-904d-45a7-ac95-a3ae526e1c43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 22 02:53:22.013517 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:22.013481 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:22.013893 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:22.013844 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:53:22.014568 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:53:22.014553 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:55:03.210758 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:03.210723 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt"] Jul 22 02:55:03.211320 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:03.210977 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" podUID="75e2b709-904d-45a7-ac95-a3ae526e1c43" containerName="istio-proxy" containerID="cri-o://f3fc5eaa8e6d028478e35b1cd1b2b76de124bf085464165dee42c831b5cbd61b" gracePeriod=30 Jul 22 02:55:07.015955 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:07.015916 2557 patch_prober.go:28] interesting pod/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 02:55:07.016335 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:07.015969 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" podUID="75e2b709-904d-45a7-ac95-a3ae526e1c43" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 02:55:08.453606 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.453572 2557 generic.go:356] "Generic (PLEG): container finished" podID="75e2b709-904d-45a7-ac95-a3ae526e1c43" containerID="f3fc5eaa8e6d028478e35b1cd1b2b76de124bf085464165dee42c831b5cbd61b" exitCode=0 Jul 22 02:55:08.454058 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.453647 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" event={"ID":"75e2b709-904d-45a7-ac95-a3ae526e1c43","Type":"ContainerDied","Data":"f3fc5eaa8e6d028478e35b1cd1b2b76de124bf085464165dee42c831b5cbd61b"} Jul 22 02:55:08.454058 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.453726 2557 scope.go:117] "RemoveContainer" containerID="52e099b837f55150b16ac56a155a61577ca10f43f43765af14ceed1b03b9127b" Jul 22 02:55:08.490557 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.490526 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:55:08.563877 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.563836 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-podinfo\") pod \"75e2b709-904d-45a7-ac95-a3ae526e1c43\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " Jul 22 02:55:08.563877 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.563884 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-envoy\") pod \"75e2b709-904d-45a7-ac95-a3ae526e1c43\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " Jul 22 02:55:08.564133 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.563912 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75e2b709-904d-45a7-ac95-a3ae526e1c43-istiod-ca-cert\") pod \"75e2b709-904d-45a7-ac95-a3ae526e1c43\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " Jul 22 02:55:08.564133 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.564092 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-workload-socket\") pod \"75e2b709-904d-45a7-ac95-a3ae526e1c43\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " Jul 22 02:55:08.564247 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.564143 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-workload-certs\") pod \"75e2b709-904d-45a7-ac95-a3ae526e1c43\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " Jul 22 02:55:08.564247 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.564169 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpl7k\" (UniqueName: \"kubernetes.io/projected/75e2b709-904d-45a7-ac95-a3ae526e1c43-kube-api-access-bpl7k\") pod \"75e2b709-904d-45a7-ac95-a3ae526e1c43\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " Jul 22 02:55:08.564247 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.564201 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-credential-socket\") pod \"75e2b709-904d-45a7-ac95-a3ae526e1c43\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " Jul 22 02:55:08.564396 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.564281 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-token\") pod \"75e2b709-904d-45a7-ac95-a3ae526e1c43\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " Jul 22 02:55:08.564396 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.564305 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-data\") pod \"75e2b709-904d-45a7-ac95-a3ae526e1c43\" (UID: \"75e2b709-904d-45a7-ac95-a3ae526e1c43\") " Jul 22 02:55:08.564396 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.564359 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "75e2b709-904d-45a7-ac95-a3ae526e1c43" (UID: "75e2b709-904d-45a7-ac95-a3ae526e1c43"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:55:08.564396 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.564371 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75e2b709-904d-45a7-ac95-a3ae526e1c43-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "75e2b709-904d-45a7-ac95-a3ae526e1c43" (UID: "75e2b709-904d-45a7-ac95-a3ae526e1c43"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:55:08.564632 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.564607 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75e2b709-904d-45a7-ac95-a3ae526e1c43-istiod-ca-cert\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:55:08.564704 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.564630 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-workload-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:55:08.564758 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.564739 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "75e2b709-904d-45a7-ac95-a3ae526e1c43" (UID: "75e2b709-904d-45a7-ac95-a3ae526e1c43"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:55:08.564946 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.564847 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "75e2b709-904d-45a7-ac95-a3ae526e1c43" (UID: "75e2b709-904d-45a7-ac95-a3ae526e1c43"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:55:08.565458 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.565433 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-data" (OuterVolumeSpecName: "istio-data") pod "75e2b709-904d-45a7-ac95-a3ae526e1c43" (UID: "75e2b709-904d-45a7-ac95-a3ae526e1c43"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:55:08.566278 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.566250 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "75e2b709-904d-45a7-ac95-a3ae526e1c43" (UID: "75e2b709-904d-45a7-ac95-a3ae526e1c43"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:55:08.566431 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.566412 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-token" (OuterVolumeSpecName: "istio-token") pod "75e2b709-904d-45a7-ac95-a3ae526e1c43" (UID: "75e2b709-904d-45a7-ac95-a3ae526e1c43"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:55:08.566643 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.566622 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "75e2b709-904d-45a7-ac95-a3ae526e1c43" (UID: "75e2b709-904d-45a7-ac95-a3ae526e1c43"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:55:08.566779 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.566760 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e2b709-904d-45a7-ac95-a3ae526e1c43-kube-api-access-bpl7k" (OuterVolumeSpecName: "kube-api-access-bpl7k") pod "75e2b709-904d-45a7-ac95-a3ae526e1c43" (UID: "75e2b709-904d-45a7-ac95-a3ae526e1c43"). InnerVolumeSpecName "kube-api-access-bpl7k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:55:08.665478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.665353 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-podinfo\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:55:08.665478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.665391 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-envoy\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:55:08.665478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.665405 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-workload-certs\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:55:08.665478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.665418 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpl7k\" (UniqueName: \"kubernetes.io/projected/75e2b709-904d-45a7-ac95-a3ae526e1c43-kube-api-access-bpl7k\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:55:08.665478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.665433 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-credential-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:55:08.665478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.665444 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-token\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:55:08.665478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:08.665455 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75e2b709-904d-45a7-ac95-a3ae526e1c43-istio-data\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:55:09.459634 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:09.459601 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" Jul 22 02:55:09.459634 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:09.459625 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt" event={"ID":"75e2b709-904d-45a7-ac95-a3ae526e1c43","Type":"ContainerDied","Data":"5eeb43bda3ab34729cb027126de1a45ca4cc016a0b312b13c27b1a44308e5305"} Jul 22 02:55:09.460205 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:09.459683 2557 scope.go:117] "RemoveContainer" containerID="f3fc5eaa8e6d028478e35b1cd1b2b76de124bf085464165dee42c831b5cbd61b" Jul 22 02:55:09.487253 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:09.487227 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt"] Jul 22 02:55:09.493535 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:09.493507 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6a145fe4-openshift-default-7cc677f5-7lqlt"] Jul 22 02:55:09.609312 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:55:09.609276 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75e2b709-904d-45a7-ac95-a3ae526e1c43" path="/var/lib/kubelet/pods/75e2b709-904d-45a7-ac95-a3ae526e1c43/volumes" Jul 22 02:56:01.057113 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.057078 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z"] Jul 22 02:56:01.057605 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.057587 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75e2b709-904d-45a7-ac95-a3ae526e1c43" containerName="istio-proxy" Jul 22 02:56:01.057605 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.057601 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e2b709-904d-45a7-ac95-a3ae526e1c43" containerName="istio-proxy" Jul 22 02:56:01.057706 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.057694 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ce54b3f-5cd8-4116-af2e-6fd175a69807" containerName="istio-proxy" Jul 22 02:56:01.057752 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.057710 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="75e2b709-904d-45a7-ac95-a3ae526e1c43" containerName="istio-proxy" Jul 22 02:56:01.057752 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.057723 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="75e2b709-904d-45a7-ac95-a3ae526e1c43" containerName="istio-proxy" Jul 22 02:56:01.061165 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.061139 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.064912 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.064889 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-fbb5cc-openshift-default-dockercfg-j97qv\"" Jul 22 02:56:01.073361 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.073340 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z"] Jul 22 02:56:01.130140 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.130104 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-data\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.130287 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.130147 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-envoy\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.130287 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.130207 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-credential-socket\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.130287 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.130234 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-token\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.130287 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.130265 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-workload-certs\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.130421 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.130289 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istiod-ca-cert\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.130421 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.130371 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59sxh\" (UniqueName: \"kubernetes.io/projected/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-kube-api-access-59sxh\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.130421 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.130411 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-workload-socket\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.130519 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.130452 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-podinfo\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.231399 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.231364 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-credential-socket\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.231399 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.231396 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-token\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.231648 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.231416 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-workload-certs\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.231648 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.231431 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istiod-ca-cert\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.231648 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.231463 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59sxh\" (UniqueName: \"kubernetes.io/projected/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-kube-api-access-59sxh\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.231648 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.231489 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-workload-socket\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.231648 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.231537 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-podinfo\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.231648 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.231597 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-data\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.231648 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.231622 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-envoy\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.232041 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.231806 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-credential-socket\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.232041 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.231806 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-workload-certs\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.232177 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.232075 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-workload-socket\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.232177 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.232096 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-data\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.232266 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.232220 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istiod-ca-cert\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.233979 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.233955 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-envoy\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.234183 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.234165 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-podinfo\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.240930 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.240905 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-token\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.241214 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.241159 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59sxh\" (UniqueName: \"kubernetes.io/projected/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-kube-api-access-59sxh\") pod \"e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.371578 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.371493 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:01.519226 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.519084 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z"] Jul 22 02:56:01.521467 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:56:01.521437 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ed2b2c6_2f0c_48fb_a7cb_da2ba4923d71.slice/crio-85c1b78e6cac1ffe33c97327852afc53d24b6bf35911b30a889b3b40df2f8213 WatchSource:0}: Error finding container 85c1b78e6cac1ffe33c97327852afc53d24b6bf35911b30a889b3b40df2f8213: Status 404 returned error can't find the container with id 85c1b78e6cac1ffe33c97327852afc53d24b6bf35911b30a889b3b40df2f8213 Jul 22 02:56:01.523623 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.523592 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:56:01.523703 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.523656 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:56:01.523703 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.523685 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:56:01.709060 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.708967 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" event={"ID":"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71","Type":"ContainerStarted","Data":"77cc17dfd18248677a43873864047543a50feffc4c6dfed9088a8d6fccb3b924"} Jul 22 02:56:01.709060 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.709023 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" event={"ID":"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71","Type":"ContainerStarted","Data":"85c1b78e6cac1ffe33c97327852afc53d24b6bf35911b30a889b3b40df2f8213"} Jul 22 02:56:01.746384 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:01.746321 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" podStartSLOduration=0.746304325 podStartE2EDuration="746.304325ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:56:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:56:01.742520326 +0000 UTC m=+2192.651385321" watchObservedRunningTime="2026-07-22 02:56:01.746304325 +0000 UTC m=+2192.655169320" Jul 22 02:56:02.371913 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:02.371876 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:02.376982 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:02.376958 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:02.714138 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:02.714109 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:02.715183 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:02.715159 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:08.964091 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:08.964058 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr"] Jul 22 02:56:08.964538 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:08.964430 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75e2b709-904d-45a7-ac95-a3ae526e1c43" containerName="istio-proxy" Jul 22 02:56:08.964538 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:08.964441 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e2b709-904d-45a7-ac95-a3ae526e1c43" containerName="istio-proxy" Jul 22 02:56:08.967583 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:08.967562 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:08.970879 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:08.970855 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-6873d4-openshift-default-dockercfg-4llnx\"" Jul 22 02:56:08.997757 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:08.997726 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr"] Jul 22 02:56:09.098478 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.098435 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-workload-certs\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.098668 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.098584 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-podinfo\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.098668 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.098637 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-credential-socket\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.098782 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.098691 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-token\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.098782 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.098743 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istiod-ca-cert\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.098896 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.098802 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-workload-socket\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.098896 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.098877 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-envoy\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.098962 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.098905 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-data\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.098962 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.098921 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5549\" (UniqueName: \"kubernetes.io/projected/f81b4eb6-df7c-49de-95fd-4a0065b58f96-kube-api-access-q5549\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.199655 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.199613 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-workload-socket\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.199655 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.199658 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-envoy\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.199883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.199685 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-data\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.199883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.199709 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5549\" (UniqueName: \"kubernetes.io/projected/f81b4eb6-df7c-49de-95fd-4a0065b58f96-kube-api-access-q5549\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.199883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.199736 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-workload-certs\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.199883 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.199796 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-podinfo\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.200095 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.199958 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-credential-socket\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.200095 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.200017 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-token\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.200095 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.200067 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istiod-ca-cert\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.200245 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.200102 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-workload-socket\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.200245 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.200200 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-data\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.200350 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.200297 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-credential-socket\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.200409 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.200384 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-workload-certs\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.200751 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.200727 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istiod-ca-cert\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.202839 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.202805 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-envoy\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.202995 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.202978 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-podinfo\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.213605 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.213578 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5549\" (UniqueName: \"kubernetes.io/projected/f81b4eb6-df7c-49de-95fd-4a0065b58f96-kube-api-access-q5549\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.217035 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.216972 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-token\") pod \"e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.278103 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.278063 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:09.423849 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.423793 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr"] Jul 22 02:56:09.426012 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:56:09.425980 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf81b4eb6_df7c_49de_95fd_4a0065b58f96.slice/crio-30f8874c1f24cd88cafed8a7a1dd3a9991ff2c27ab117ab82f00e77aea247ab6 WatchSource:0}: Error finding container 30f8874c1f24cd88cafed8a7a1dd3a9991ff2c27ab117ab82f00e77aea247ab6: Status 404 returned error can't find the container with id 30f8874c1f24cd88cafed8a7a1dd3a9991ff2c27ab117ab82f00e77aea247ab6 Jul 22 02:56:09.428186 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.428154 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:56:09.428259 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.428221 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:56:09.428259 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.428248 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:56:09.748231 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.748134 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" event={"ID":"f81b4eb6-df7c-49de-95fd-4a0065b58f96","Type":"ContainerStarted","Data":"917d2b8c5b8eb1b18913401385036b1683c41a06abd06d796aa6b5983b1fcaff"} Jul 22 02:56:09.748231 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.748188 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" event={"ID":"f81b4eb6-df7c-49de-95fd-4a0065b58f96","Type":"ContainerStarted","Data":"30f8874c1f24cd88cafed8a7a1dd3a9991ff2c27ab117ab82f00e77aea247ab6"} Jul 22 02:56:09.774532 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:09.774481 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" podStartSLOduration=1.774465028 podStartE2EDuration="1.774465028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:56:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:56:09.771157689 +0000 UTC m=+2200.680022683" watchObservedRunningTime="2026-07-22 02:56:09.774465028 +0000 UTC m=+2200.683330022" Jul 22 02:56:10.279215 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:10.279178 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:10.284636 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:10.284608 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:10.752649 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:10.752608 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:10.753725 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:10.753701 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:17.877237 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:17.875800 2557 patch_prober.go:28] interesting pod/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": read tcp 10.133.0.2:57420->10.133.0.48:15021: read: connection reset by peer" start-of-body= Jul 22 02:56:17.877237 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:17.875891 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" podUID="4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": read tcp 10.133.0.2:57420->10.133.0.48:15021: read: connection reset by peer" Jul 22 02:56:18.798305 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:18.798265 2557 generic.go:356] "Generic (PLEG): container finished" podID="4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" containerID="77cc17dfd18248677a43873864047543a50feffc4c6dfed9088a8d6fccb3b924" exitCode=0 Jul 22 02:56:18.798641 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:18.798614 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" event={"ID":"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71","Type":"ContainerDied","Data":"77cc17dfd18248677a43873864047543a50feffc4c6dfed9088a8d6fccb3b924"} Jul 22 02:56:18.799252 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:18.799231 2557 scope.go:117] "RemoveContainer" containerID="77cc17dfd18248677a43873864047543a50feffc4c6dfed9088a8d6fccb3b924" Jul 22 02:56:18.801307 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:18.801273 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:56:18.801489 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:18.801468 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:56:18.801629 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:18.801612 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:56:19.809185 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:19.809134 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" event={"ID":"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71","Type":"ContainerStarted","Data":"1fd49805e5d0c1d88bb1fa8e44539a3fb84bdbea51200a923987a6ecbd80faee"} Jul 22 02:56:20.372539 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:20.372503 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:20.373196 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:20.372908 2557 patch_prober.go:28] interesting pod/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 02:56:20.373196 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:20.372959 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" podUID="4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 02:56:21.372460 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:21.372420 2557 patch_prober.go:28] interesting pod/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 02:56:21.372976 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:21.372479 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" podUID="4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 02:56:22.372195 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:22.372158 2557 patch_prober.go:28] interesting pod/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 02:56:22.372390 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:22.372219 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" podUID="4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 02:56:23.371917 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:23.371880 2557 patch_prober.go:28] interesting pod/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 02:56:23.372323 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:23.371935 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" podUID="4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 02:56:24.376034 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:24.376005 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:24.376476 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:24.376277 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:24.377097 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:24.377079 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:56:25.754209 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:25.754172 2557 patch_prober.go:28] interesting pod/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 22 02:56:25.754622 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:25.754227 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" podUID="f81b4eb6-df7c-49de-95fd-4a0065b58f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 22 02:56:25.842404 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:25.842369 2557 generic.go:356] "Generic (PLEG): container finished" podID="f81b4eb6-df7c-49de-95fd-4a0065b58f96" containerID="917d2b8c5b8eb1b18913401385036b1683c41a06abd06d796aa6b5983b1fcaff" exitCode=0 Jul 22 02:56:25.842582 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:25.842442 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" event={"ID":"f81b4eb6-df7c-49de-95fd-4a0065b58f96","Type":"ContainerDied","Data":"917d2b8c5b8eb1b18913401385036b1683c41a06abd06d796aa6b5983b1fcaff"} Jul 22 02:56:25.843062 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:25.843044 2557 scope.go:117] "RemoveContainer" containerID="917d2b8c5b8eb1b18913401385036b1683c41a06abd06d796aa6b5983b1fcaff" Jul 22 02:56:25.844025 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:25.843996 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:56:25.844117 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:25.844062 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:56:25.844117 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:25.844094 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:56:26.848927 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:26.848889 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" event={"ID":"f81b4eb6-df7c-49de-95fd-4a0065b58f96","Type":"ContainerStarted","Data":"6d0006f08198b44f07bfc40031860f230a85f1a704ba4eb6d0e9dfa8d4db816b"} Jul 22 02:56:27.278532 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:27.278490 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:27.278720 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:27.278677 2557 patch_prober.go:28] interesting pod/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 22 02:56:27.278780 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:27.278734 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" podUID="f81b4eb6-df7c-49de-95fd-4a0065b58f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 22 02:56:28.279374 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:28.279334 2557 patch_prober.go:28] interesting pod/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 22 02:56:28.279759 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:28.279397 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" podUID="f81b4eb6-df7c-49de-95fd-4a0065b58f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 22 02:56:29.279030 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:29.278993 2557 patch_prober.go:28] interesting pod/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 22 02:56:29.279214 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:29.279049 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" podUID="f81b4eb6-df7c-49de-95fd-4a0065b58f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 22 02:56:30.279285 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:30.279244 2557 patch_prober.go:28] interesting pod/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 22 02:56:30.279654 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:30.279308 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" podUID="f81b4eb6-df7c-49de-95fd-4a0065b58f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 22 02:56:31.282535 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:31.282505 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:31.283035 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:31.282778 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:56:31.283487 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:56:31.283467 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:58:14.810490 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:14.810453 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z"] Jul 22 02:58:14.810992 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:14.810700 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" podUID="4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" containerName="istio-proxy" containerID="cri-o://1fd49805e5d0c1d88bb1fa8e44539a3fb84bdbea51200a923987a6ecbd80faee" gracePeriod=30 Jul 22 02:58:20.067344 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.067317 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:58:20.116055 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.115965 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-workload-socket\") pod \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " Jul 22 02:58:20.116055 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.116010 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-podinfo\") pod \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " Jul 22 02:58:20.116055 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.116027 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-token\") pod \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " Jul 22 02:58:20.116055 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.116050 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istiod-ca-cert\") pod \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " Jul 22 02:58:20.116416 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.116105 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-workload-certs\") pod \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " Jul 22 02:58:20.116416 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.116126 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-data\") pod \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " Jul 22 02:58:20.116416 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.116154 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-envoy\") pod \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " Jul 22 02:58:20.116416 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.116180 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59sxh\" (UniqueName: \"kubernetes.io/projected/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-kube-api-access-59sxh\") pod \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " Jul 22 02:58:20.116416 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.116204 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-credential-socket\") pod \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\" (UID: \"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71\") " Jul 22 02:58:20.116416 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.116272 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" (UID: "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:20.116744 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.116495 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-workload-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:20.116744 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.116553 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" (UID: "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:20.116744 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.116673 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" (UID: "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:20.116913 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.116739 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" (UID: "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:58:20.117218 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.117195 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-data" (OuterVolumeSpecName: "istio-data") pod "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" (UID: "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:20.118501 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.118470 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-token" (OuterVolumeSpecName: "istio-token") pod "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" (UID: "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:58:20.118608 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.118583 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" (UID: "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:20.119002 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.118978 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-kube-api-access-59sxh" (OuterVolumeSpecName: "kube-api-access-59sxh") pod "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" (UID: "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71"). InnerVolumeSpecName "kube-api-access-59sxh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:58:20.119075 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.119016 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" (UID: "4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:58:20.217084 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.217041 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-envoy\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:20.217084 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.217083 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59sxh\" (UniqueName: \"kubernetes.io/projected/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-kube-api-access-59sxh\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:20.217291 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.217099 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-credential-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:20.217291 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.217111 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-podinfo\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:20.217291 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.217120 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-token\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:20.217291 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.217128 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istiod-ca-cert\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:20.217291 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.217135 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-workload-certs\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:20.217291 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.217142 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71-istio-data\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:20.404289 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.404189 2557 generic.go:356] "Generic (PLEG): container finished" podID="4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" containerID="1fd49805e5d0c1d88bb1fa8e44539a3fb84bdbea51200a923987a6ecbd80faee" exitCode=0 Jul 22 02:58:20.404289 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.404229 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" event={"ID":"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71","Type":"ContainerDied","Data":"1fd49805e5d0c1d88bb1fa8e44539a3fb84bdbea51200a923987a6ecbd80faee"} Jul 22 02:58:20.404289 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.404287 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" event={"ID":"4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71","Type":"ContainerDied","Data":"85c1b78e6cac1ffe33c97327852afc53d24b6bf35911b30a889b3b40df2f8213"} Jul 22 02:58:20.404519 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.404310 2557 scope.go:117] "RemoveContainer" containerID="1fd49805e5d0c1d88bb1fa8e44539a3fb84bdbea51200a923987a6ecbd80faee" Jul 22 02:58:20.404519 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.404242 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z" Jul 22 02:58:20.413250 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.413233 2557 scope.go:117] "RemoveContainer" containerID="77cc17dfd18248677a43873864047543a50feffc4c6dfed9088a8d6fccb3b924" Jul 22 02:58:20.421309 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.421291 2557 scope.go:117] "RemoveContainer" containerID="1fd49805e5d0c1d88bb1fa8e44539a3fb84bdbea51200a923987a6ecbd80faee" Jul 22 02:58:20.421578 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:58:20.421560 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fd49805e5d0c1d88bb1fa8e44539a3fb84bdbea51200a923987a6ecbd80faee\": container with ID starting with 1fd49805e5d0c1d88bb1fa8e44539a3fb84bdbea51200a923987a6ecbd80faee not found: ID does not exist" containerID="1fd49805e5d0c1d88bb1fa8e44539a3fb84bdbea51200a923987a6ecbd80faee" Jul 22 02:58:20.421629 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.421588 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fd49805e5d0c1d88bb1fa8e44539a3fb84bdbea51200a923987a6ecbd80faee"} err="failed to get container status \"1fd49805e5d0c1d88bb1fa8e44539a3fb84bdbea51200a923987a6ecbd80faee\": rpc error: code = NotFound desc = could not find container \"1fd49805e5d0c1d88bb1fa8e44539a3fb84bdbea51200a923987a6ecbd80faee\": container with ID starting with 1fd49805e5d0c1d88bb1fa8e44539a3fb84bdbea51200a923987a6ecbd80faee not found: ID does not exist" Jul 22 02:58:20.421629 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.421604 2557 scope.go:117] "RemoveContainer" containerID="77cc17dfd18248677a43873864047543a50feffc4c6dfed9088a8d6fccb3b924" Jul 22 02:58:20.421860 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:58:20.421843 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77cc17dfd18248677a43873864047543a50feffc4c6dfed9088a8d6fccb3b924\": container with ID starting with 77cc17dfd18248677a43873864047543a50feffc4c6dfed9088a8d6fccb3b924 not found: ID does not exist" containerID="77cc17dfd18248677a43873864047543a50feffc4c6dfed9088a8d6fccb3b924" Jul 22 02:58:20.421911 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.421866 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77cc17dfd18248677a43873864047543a50feffc4c6dfed9088a8d6fccb3b924"} err="failed to get container status \"77cc17dfd18248677a43873864047543a50feffc4c6dfed9088a8d6fccb3b924\": rpc error: code = NotFound desc = could not find container \"77cc17dfd18248677a43873864047543a50feffc4c6dfed9088a8d6fccb3b924\": container with ID starting with 77cc17dfd18248677a43873864047543a50feffc4c6dfed9088a8d6fccb3b924 not found: ID does not exist" Jul 22 02:58:20.427771 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.427747 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z"] Jul 22 02:58:20.432281 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:20.432252 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-fbb5cc-openshift-default-698d4f6fff-k9f8z"] Jul 22 02:58:21.614643 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:21.611903 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" path="/var/lib/kubelet/pods/4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71/volumes" Jul 22 02:58:37.260711 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:37.260673 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr"] Jul 22 02:58:37.261191 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:37.260908 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" podUID="f81b4eb6-df7c-49de-95fd-4a0065b58f96" containerName="istio-proxy" containerID="cri-o://6d0006f08198b44f07bfc40031860f230a85f1a704ba4eb6d0e9dfa8d4db816b" gracePeriod=30 Jul 22 02:58:38.177443 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.177408 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g"] Jul 22 02:58:38.177880 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.177862 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" containerName="istio-proxy" Jul 22 02:58:38.177986 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.177882 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" containerName="istio-proxy" Jul 22 02:58:38.177986 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.177893 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" containerName="istio-proxy" Jul 22 02:58:38.177986 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.177902 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" containerName="istio-proxy" Jul 22 02:58:38.178146 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.177989 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" containerName="istio-proxy" Jul 22 02:58:38.180075 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.180053 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.184118 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.184089 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-3f8fef-openshift-default-dockercfg-fpk78\"" Jul 22 02:58:38.197297 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.197270 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g"] Jul 22 02:58:38.272591 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.272549 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-workload-certs\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.273053 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.272665 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-workload-socket\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.273053 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.272707 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-podinfo\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.273053 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.272764 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnxmx\" (UniqueName: \"kubernetes.io/projected/7cf5a8bd-2475-4def-b4bc-364e413683dc-kube-api-access-cnxmx\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.273053 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.272805 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-data\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.273053 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.272858 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cf5a8bd-2475-4def-b4bc-364e413683dc-istiod-ca-cert\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.273053 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.272953 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-credential-socket\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.273053 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.273008 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-token\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.273053 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.273049 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-envoy\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.374070 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.374035 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-workload-certs\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.374278 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.374087 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-workload-socket\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.374278 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.374212 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-podinfo\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.374278 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.374257 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnxmx\" (UniqueName: \"kubernetes.io/projected/7cf5a8bd-2475-4def-b4bc-364e413683dc-kube-api-access-cnxmx\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.374449 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.374287 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-data\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.374449 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.374323 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cf5a8bd-2475-4def-b4bc-364e413683dc-istiod-ca-cert\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.374449 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.374409 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-credential-socket\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.374611 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.374491 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-workload-socket\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.374611 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.374535 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-workload-certs\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.374743 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.374719 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-data\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.374804 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.374767 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-credential-socket\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.375133 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.375108 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cf5a8bd-2475-4def-b4bc-364e413683dc-istiod-ca-cert\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.375227 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.375214 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-token\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.375267 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.375253 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-envoy\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.377098 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.377071 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-podinfo\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.377446 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.377427 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-envoy\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.384246 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.384199 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnxmx\" (UniqueName: \"kubernetes.io/projected/7cf5a8bd-2475-4def-b4bc-364e413683dc-kube-api-access-cnxmx\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.386790 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.386765 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-token\") pod \"e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.492943 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.492914 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:38.642272 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.642240 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g"] Jul 22 02:58:38.643579 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:58:38.643553 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cf5a8bd_2475_4def_b4bc_364e413683dc.slice/crio-9b4da695d77287f7855358ac89b2445755494615c848516df53a733ae38d7f64 WatchSource:0}: Error finding container 9b4da695d77287f7855358ac89b2445755494615c848516df53a733ae38d7f64: Status 404 returned error can't find the container with id 9b4da695d77287f7855358ac89b2445755494615c848516df53a733ae38d7f64 Jul 22 02:58:38.645649 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.645618 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:58:38.645811 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.645691 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:58:38.645811 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:38.645734 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:58:39.497648 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:39.497610 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" event={"ID":"7cf5a8bd-2475-4def-b4bc-364e413683dc","Type":"ContainerStarted","Data":"1e44fd1264e16e1db549993ccea5312422456f76289f687fddfab04aaaefbdab"} Jul 22 02:58:39.498137 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:39.497679 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" event={"ID":"7cf5a8bd-2475-4def-b4bc-364e413683dc","Type":"ContainerStarted","Data":"9b4da695d77287f7855358ac89b2445755494615c848516df53a733ae38d7f64"} Jul 22 02:58:39.526380 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:39.526334 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" podStartSLOduration=1.526317266 podStartE2EDuration="1.526317266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:58:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:58:39.525222722 +0000 UTC m=+2350.434087718" watchObservedRunningTime="2026-07-22 02:58:39.526317266 +0000 UTC m=+2350.435182319" Jul 22 02:58:40.493911 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:40.493871 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:40.499539 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:40.499514 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:40.502884 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:40.502860 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:40.503677 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:40.503658 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:42.513564 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.513530 2557 generic.go:356] "Generic (PLEG): container finished" podID="f81b4eb6-df7c-49de-95fd-4a0065b58f96" containerID="6d0006f08198b44f07bfc40031860f230a85f1a704ba4eb6d0e9dfa8d4db816b" exitCode=0 Jul 22 02:58:42.513931 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.513614 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" event={"ID":"f81b4eb6-df7c-49de-95fd-4a0065b58f96","Type":"ContainerDied","Data":"6d0006f08198b44f07bfc40031860f230a85f1a704ba4eb6d0e9dfa8d4db816b"} Jul 22 02:58:42.513931 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.513680 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" event={"ID":"f81b4eb6-df7c-49de-95fd-4a0065b58f96","Type":"ContainerDied","Data":"30f8874c1f24cd88cafed8a7a1dd3a9991ff2c27ab117ab82f00e77aea247ab6"} Jul 22 02:58:42.513931 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.513699 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="30f8874c1f24cd88cafed8a7a1dd3a9991ff2c27ab117ab82f00e77aea247ab6" Jul 22 02:58:42.513931 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.513716 2557 scope.go:117] "RemoveContainer" containerID="917d2b8c5b8eb1b18913401385036b1683c41a06abd06d796aa6b5983b1fcaff" Jul 22 02:58:42.532517 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:58:42.532483 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"917d2b8c5b8eb1b18913401385036b1683c41a06abd06d796aa6b5983b1fcaff\": container with ID starting with 917d2b8c5b8eb1b18913401385036b1683c41a06abd06d796aa6b5983b1fcaff not found: ID does not exist" containerID="917d2b8c5b8eb1b18913401385036b1683c41a06abd06d796aa6b5983b1fcaff" Jul 22 02:58:42.534533 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.534505 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:58:42.610393 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.610365 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-podinfo\") pod \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " Jul 22 02:58:42.610569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.610426 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-token\") pod \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " Jul 22 02:58:42.610569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.610446 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5549\" (UniqueName: \"kubernetes.io/projected/f81b4eb6-df7c-49de-95fd-4a0065b58f96-kube-api-access-q5549\") pod \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " Jul 22 02:58:42.610569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.610483 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-workload-certs\") pod \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " Jul 22 02:58:42.610569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.610509 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istiod-ca-cert\") pod \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " Jul 22 02:58:42.610569 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.610549 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-credential-socket\") pod \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " Jul 22 02:58:42.610872 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.610576 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-workload-socket\") pod \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " Jul 22 02:58:42.610872 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.610624 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-data\") pod \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " Jul 22 02:58:42.610872 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.610678 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-envoy\") pod \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\" (UID: \"f81b4eb6-df7c-49de-95fd-4a0065b58f96\") " Jul 22 02:58:42.610872 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.610760 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f81b4eb6-df7c-49de-95fd-4a0065b58f96" (UID: "f81b4eb6-df7c-49de-95fd-4a0065b58f96"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:42.611068 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.610886 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f81b4eb6-df7c-49de-95fd-4a0065b58f96" (UID: "f81b4eb6-df7c-49de-95fd-4a0065b58f96"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:58:42.611068 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.611027 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f81b4eb6-df7c-49de-95fd-4a0065b58f96" (UID: "f81b4eb6-df7c-49de-95fd-4a0065b58f96"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:42.611367 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.611347 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-workload-certs\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:42.611509 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.611485 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f81b4eb6-df7c-49de-95fd-4a0065b58f96" (UID: "f81b4eb6-df7c-49de-95fd-4a0065b58f96"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:42.611585 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.611546 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istiod-ca-cert\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:42.611585 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.611566 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-workload-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:42.611988 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.611965 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-data" (OuterVolumeSpecName: "istio-data") pod "f81b4eb6-df7c-49de-95fd-4a0065b58f96" (UID: "f81b4eb6-df7c-49de-95fd-4a0065b58f96"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:42.612843 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.612797 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f81b4eb6-df7c-49de-95fd-4a0065b58f96" (UID: "f81b4eb6-df7c-49de-95fd-4a0065b58f96"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:58:42.612923 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.612911 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-token" (OuterVolumeSpecName: "istio-token") pod "f81b4eb6-df7c-49de-95fd-4a0065b58f96" (UID: "f81b4eb6-df7c-49de-95fd-4a0065b58f96"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:58:42.613310 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.613285 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f81b4eb6-df7c-49de-95fd-4a0065b58f96-kube-api-access-q5549" (OuterVolumeSpecName: "kube-api-access-q5549") pod "f81b4eb6-df7c-49de-95fd-4a0065b58f96" (UID: "f81b4eb6-df7c-49de-95fd-4a0065b58f96"). InnerVolumeSpecName "kube-api-access-q5549". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:58:42.613310 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.613290 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f81b4eb6-df7c-49de-95fd-4a0065b58f96" (UID: "f81b4eb6-df7c-49de-95fd-4a0065b58f96"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:42.712612 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.712577 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-credential-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:42.712612 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.712605 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-data\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:42.712612 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.712614 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-envoy\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:42.712845 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.712622 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-podinfo\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:42.712845 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.712632 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f81b4eb6-df7c-49de-95fd-4a0065b58f96-istio-token\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:42.712845 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:42.712640 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5549\" (UniqueName: \"kubernetes.io/projected/f81b4eb6-df7c-49de-95fd-4a0065b58f96-kube-api-access-q5549\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:58:43.519362 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:43.519332 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr" Jul 22 02:58:43.541561 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:43.541535 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr"] Jul 22 02:58:43.546092 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:43.546069 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-6873d4-openshift-default-5f8ccf9cbc-t9zpr"] Jul 22 02:58:43.608951 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:43.608921 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f81b4eb6-df7c-49de-95fd-4a0065b58f96" path="/var/lib/kubelet/pods/f81b4eb6-df7c-49de-95fd-4a0065b58f96/volumes" Jul 22 02:58:46.769139 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.769104 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp"] Jul 22 02:58:46.769574 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.769477 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f81b4eb6-df7c-49de-95fd-4a0065b58f96" containerName="istio-proxy" Jul 22 02:58:46.769574 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.769489 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f81b4eb6-df7c-49de-95fd-4a0065b58f96" containerName="istio-proxy" Jul 22 02:58:46.769574 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.769497 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f81b4eb6-df7c-49de-95fd-4a0065b58f96" containerName="istio-proxy" Jul 22 02:58:46.769574 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.769503 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f81b4eb6-df7c-49de-95fd-4a0065b58f96" containerName="istio-proxy" Jul 22 02:58:46.769721 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.769578 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f81b4eb6-df7c-49de-95fd-4a0065b58f96" containerName="istio-proxy" Jul 22 02:58:46.769721 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.769587 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ed2b2c6-2f0c-48fb-a7cb-da2ba4923d71" containerName="istio-proxy" Jul 22 02:58:46.769721 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.769595 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f81b4eb6-df7c-49de-95fd-4a0065b58f96" containerName="istio-proxy" Jul 22 02:58:46.774245 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.774225 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.777072 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.777052 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-dac71a-openshift-default-dockercfg-nhrww\"" Jul 22 02:58:46.788715 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.788690 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp"] Jul 22 02:58:46.849678 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.849640 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-workload-certs\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.849877 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.849688 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-workload-socket\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.849877 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.849715 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-envoy\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.849877 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.849760 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-data\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.849877 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.849802 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istiod-ca-cert\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.850030 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.849881 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-credential-socket\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.850030 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.849907 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-token\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.850030 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.849932 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-668c5\" (UniqueName: \"kubernetes.io/projected/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-kube-api-access-668c5\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.850125 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.850027 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-podinfo\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.951067 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.951028 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-credential-socket\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.951067 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.951069 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-token\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.951316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.951086 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-668c5\" (UniqueName: \"kubernetes.io/projected/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-kube-api-access-668c5\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.951316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.951124 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-podinfo\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.951316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.951171 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-workload-certs\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.951316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.951192 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-workload-socket\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.951316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.951216 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-envoy\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.951316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.951268 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-data\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.951316 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.951311 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istiod-ca-cert\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.951663 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.951515 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-credential-socket\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.951663 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.951563 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-workload-certs\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.951734 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.951716 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-workload-socket\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.951806 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.951791 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-data\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.952278 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.952255 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istiod-ca-cert\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.953864 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.953838 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-envoy\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.953970 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.953912 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-podinfo\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.960191 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.960167 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-token\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:46.960289 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:46.960233 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-668c5\" (UniqueName: \"kubernetes.io/projected/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-kube-api-access-668c5\") pod \"e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:47.085087 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:47.084976 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:47.254138 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:47.254103 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp"] Jul 22 02:58:47.255998 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:58:47.255970 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fe80d4e_f4fd_40b6_a6b0_090717bba83d.slice/crio-a49ee879dc39c7d1e58ce42003e2f24352d62101080a964b48945d2729a2f34b WatchSource:0}: Error finding container a49ee879dc39c7d1e58ce42003e2f24352d62101080a964b48945d2729a2f34b: Status 404 returned error can't find the container with id a49ee879dc39c7d1e58ce42003e2f24352d62101080a964b48945d2729a2f34b Jul 22 02:58:47.258267 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:47.258233 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:58:47.258351 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:47.258298 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:58:47.258351 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:47.258324 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:58:47.540589 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:47.540557 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" event={"ID":"1fe80d4e-f4fd-40b6-a6b0-090717bba83d","Type":"ContainerStarted","Data":"7f701715f1803ead853d11e874db8a6f139c772416b9f4b6db907893086db40c"} Jul 22 02:58:47.540771 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:47.540612 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" event={"ID":"1fe80d4e-f4fd-40b6-a6b0-090717bba83d","Type":"ContainerStarted","Data":"a49ee879dc39c7d1e58ce42003e2f24352d62101080a964b48945d2729a2f34b"} Jul 22 02:58:47.568018 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:47.567961 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" podStartSLOduration=1.567945932 podStartE2EDuration="1.567945932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:58:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:58:47.566072255 +0000 UTC m=+2358.474937252" watchObservedRunningTime="2026-07-22 02:58:47.567945932 +0000 UTC m=+2358.476810927" Jul 22 02:58:48.085522 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:48.085484 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:48.090734 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:48.090700 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:48.545106 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:48.545079 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:48.546014 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:48.545996 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:58:56.544023 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:56.543975 2557 patch_prober.go:28] interesting pod/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 02:58:56.544404 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:56.544037 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" podUID="7cf5a8bd-2475-4def-b4bc-364e413683dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 02:58:57.594467 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:57.594429 2557 generic.go:356] "Generic (PLEG): container finished" podID="7cf5a8bd-2475-4def-b4bc-364e413683dc" containerID="1e44fd1264e16e1db549993ccea5312422456f76289f687fddfab04aaaefbdab" exitCode=0 Jul 22 02:58:57.594912 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:57.594507 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" event={"ID":"7cf5a8bd-2475-4def-b4bc-364e413683dc","Type":"ContainerDied","Data":"1e44fd1264e16e1db549993ccea5312422456f76289f687fddfab04aaaefbdab"} Jul 22 02:58:57.594988 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:57.594973 2557 scope.go:117] "RemoveContainer" containerID="1e44fd1264e16e1db549993ccea5312422456f76289f687fddfab04aaaefbdab" Jul 22 02:58:57.596132 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:57.596101 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:58:57.596215 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:57.596181 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:58:57.596255 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:57.596215 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:58:58.605177 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:58.605127 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" event={"ID":"7cf5a8bd-2475-4def-b4bc-364e413683dc","Type":"ContainerStarted","Data":"39cc7c0c71ffe899064f1a305b4ad83ddffe6785502c76025157bfe7776bd0e9"} Jul 22 02:58:59.494029 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:59.493984 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:58:59.494253 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:59.494128 2557 patch_prober.go:28] interesting pod/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 22 02:58:59.494253 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:58:59.494175 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" podUID="7cf5a8bd-2475-4def-b4bc-364e413683dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 22 02:59:00.494143 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:00.494091 2557 patch_prober.go:28] interesting pod/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 22 02:59:00.494703 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:00.494180 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" podUID="7cf5a8bd-2475-4def-b4bc-364e413683dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 22 02:59:01.493665 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:01.493623 2557 patch_prober.go:28] interesting pod/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 22 02:59:01.493990 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:01.493954 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" podUID="7cf5a8bd-2475-4def-b4bc-364e413683dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 22 02:59:02.494383 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:02.494337 2557 patch_prober.go:28] interesting pod/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 22 02:59:02.494905 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:02.494410 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" podUID="7cf5a8bd-2475-4def-b4bc-364e413683dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 22 02:59:03.498483 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:03.498446 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:59:03.498955 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:03.498934 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:59:03.499794 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:03.499774 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:59:04.587003 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:04.586954 2557 patch_prober.go:28] interesting pod/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 02:59:04.587512 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:04.587034 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" podUID="1fe80d4e-f4fd-40b6-a6b0-090717bba83d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 02:59:05.651961 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:05.651877 2557 generic.go:356] "Generic (PLEG): container finished" podID="1fe80d4e-f4fd-40b6-a6b0-090717bba83d" containerID="7f701715f1803ead853d11e874db8a6f139c772416b9f4b6db907893086db40c" exitCode=0 Jul 22 02:59:05.651961 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:05.651952 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" event={"ID":"1fe80d4e-f4fd-40b6-a6b0-090717bba83d","Type":"ContainerDied","Data":"7f701715f1803ead853d11e874db8a6f139c772416b9f4b6db907893086db40c"} Jul 22 02:59:05.652449 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:05.652337 2557 scope.go:117] "RemoveContainer" containerID="7f701715f1803ead853d11e874db8a6f139c772416b9f4b6db907893086db40c" Jul 22 02:59:05.653327 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:05.653297 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:59:05.653381 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:05.653365 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:59:05.653417 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:05.653394 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:59:06.657905 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:06.657871 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" event={"ID":"1fe80d4e-f4fd-40b6-a6b0-090717bba83d","Type":"ContainerStarted","Data":"5b95126dde347d009e4067389795842f492e71c9b11cf26531a11848dbaef275"} Jul 22 02:59:07.085407 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:07.085375 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:59:07.085797 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:07.085718 2557 patch_prober.go:28] interesting pod/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 22 02:59:07.085982 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:07.085961 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" podUID="1fe80d4e-f4fd-40b6-a6b0-090717bba83d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 22 02:59:08.085852 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:08.085784 2557 patch_prober.go:28] interesting pod/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 22 02:59:08.086346 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:08.085879 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" podUID="1fe80d4e-f4fd-40b6-a6b0-090717bba83d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 22 02:59:09.086158 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:09.085693 2557 patch_prober.go:28] interesting pod/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 22 02:59:09.086158 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:09.085760 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" podUID="1fe80d4e-f4fd-40b6-a6b0-090717bba83d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 22 02:59:10.085562 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:10.085520 2557 patch_prober.go:28] interesting pod/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 22 02:59:10.085868 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:10.085813 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" podUID="1fe80d4e-f4fd-40b6-a6b0-090717bba83d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 22 02:59:11.089942 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:11.089910 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:59:11.090411 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:11.090302 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:59:11.091148 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:11.091126 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 02:59:29.352671 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:29.352637 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g"] Jul 22 02:59:29.353189 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:29.352871 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" podUID="7cf5a8bd-2475-4def-b4bc-364e413683dc" containerName="istio-proxy" containerID="cri-o://39cc7c0c71ffe899064f1a305b4ad83ddffe6785502c76025157bfe7776bd0e9" gracePeriod=30 Jul 22 02:59:33.500877 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:33.500841 2557 patch_prober.go:28] interesting pod/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 02:59:33.501261 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:33.500900 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" podUID="7cf5a8bd-2475-4def-b4bc-364e413683dc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 02:59:34.604618 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.604592 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:59:34.617124 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.617060 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-podinfo\") pod \"7cf5a8bd-2475-4def-b4bc-364e413683dc\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " Jul 22 02:59:34.617124 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.617102 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-workload-socket\") pod \"7cf5a8bd-2475-4def-b4bc-364e413683dc\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " Jul 22 02:59:34.617308 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.617131 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnxmx\" (UniqueName: \"kubernetes.io/projected/7cf5a8bd-2475-4def-b4bc-364e413683dc-kube-api-access-cnxmx\") pod \"7cf5a8bd-2475-4def-b4bc-364e413683dc\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " Jul 22 02:59:34.617308 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.617193 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-token\") pod \"7cf5a8bd-2475-4def-b4bc-364e413683dc\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " Jul 22 02:59:34.617308 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.617244 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-credential-socket\") pod \"7cf5a8bd-2475-4def-b4bc-364e413683dc\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " Jul 22 02:59:34.617308 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.617276 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cf5a8bd-2475-4def-b4bc-364e413683dc-istiod-ca-cert\") pod \"7cf5a8bd-2475-4def-b4bc-364e413683dc\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " Jul 22 02:59:34.617507 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.617314 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-workload-certs\") pod \"7cf5a8bd-2475-4def-b4bc-364e413683dc\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " Jul 22 02:59:34.617507 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.617329 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7cf5a8bd-2475-4def-b4bc-364e413683dc" (UID: "7cf5a8bd-2475-4def-b4bc-364e413683dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:59:34.617507 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.617420 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-data\") pod \"7cf5a8bd-2475-4def-b4bc-364e413683dc\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " Jul 22 02:59:34.617665 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.617508 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-envoy\") pod \"7cf5a8bd-2475-4def-b4bc-364e413683dc\" (UID: \"7cf5a8bd-2475-4def-b4bc-364e413683dc\") " Jul 22 02:59:34.617665 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.617637 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7cf5a8bd-2475-4def-b4bc-364e413683dc" (UID: "7cf5a8bd-2475-4def-b4bc-364e413683dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:59:34.617785 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.617754 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cf5a8bd-2475-4def-b4bc-364e413683dc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7cf5a8bd-2475-4def-b4bc-364e413683dc" (UID: "7cf5a8bd-2475-4def-b4bc-364e413683dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:59:34.617904 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.617865 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7cf5a8bd-2475-4def-b4bc-364e413683dc" (UID: "7cf5a8bd-2475-4def-b4bc-364e413683dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:59:34.618009 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.617993 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-credential-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:59:34.618078 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.618014 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cf5a8bd-2475-4def-b4bc-364e413683dc-istiod-ca-cert\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:59:34.618078 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.618027 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-workload-certs\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:59:34.618078 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.618039 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-workload-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:59:34.618313 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.618289 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-data" (OuterVolumeSpecName: "istio-data") pod "7cf5a8bd-2475-4def-b4bc-364e413683dc" (UID: "7cf5a8bd-2475-4def-b4bc-364e413683dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:59:34.620010 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.619986 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7cf5a8bd-2475-4def-b4bc-364e413683dc" (UID: "7cf5a8bd-2475-4def-b4bc-364e413683dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:59:34.620121 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.620029 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cf5a8bd-2475-4def-b4bc-364e413683dc-kube-api-access-cnxmx" (OuterVolumeSpecName: "kube-api-access-cnxmx") pod "7cf5a8bd-2475-4def-b4bc-364e413683dc" (UID: "7cf5a8bd-2475-4def-b4bc-364e413683dc"). InnerVolumeSpecName "kube-api-access-cnxmx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:59:34.620171 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.620120 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-token" (OuterVolumeSpecName: "istio-token") pod "7cf5a8bd-2475-4def-b4bc-364e413683dc" (UID: "7cf5a8bd-2475-4def-b4bc-364e413683dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:59:34.620171 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.620124 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7cf5a8bd-2475-4def-b4bc-364e413683dc" (UID: "7cf5a8bd-2475-4def-b4bc-364e413683dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:59:34.718658 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.718615 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-data\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:59:34.718658 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.718653 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-envoy\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:59:34.718658 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.718662 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-podinfo\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:59:34.718658 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.718672 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnxmx\" (UniqueName: \"kubernetes.io/projected/7cf5a8bd-2475-4def-b4bc-364e413683dc-kube-api-access-cnxmx\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:59:34.718962 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.718681 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cf5a8bd-2475-4def-b4bc-364e413683dc-istio-token\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 02:59:34.799863 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.799797 2557 generic.go:356] "Generic (PLEG): container finished" podID="7cf5a8bd-2475-4def-b4bc-364e413683dc" containerID="39cc7c0c71ffe899064f1a305b4ad83ddffe6785502c76025157bfe7776bd0e9" exitCode=0 Jul 22 02:59:34.800016 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.799869 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" Jul 22 02:59:34.800016 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.799896 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" event={"ID":"7cf5a8bd-2475-4def-b4bc-364e413683dc","Type":"ContainerDied","Data":"39cc7c0c71ffe899064f1a305b4ad83ddffe6785502c76025157bfe7776bd0e9"} Jul 22 02:59:34.800016 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.799958 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g" event={"ID":"7cf5a8bd-2475-4def-b4bc-364e413683dc","Type":"ContainerDied","Data":"9b4da695d77287f7855358ac89b2445755494615c848516df53a733ae38d7f64"} Jul 22 02:59:34.800016 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.799981 2557 scope.go:117] "RemoveContainer" containerID="39cc7c0c71ffe899064f1a305b4ad83ddffe6785502c76025157bfe7776bd0e9" Jul 22 02:59:34.809338 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.809321 2557 scope.go:117] "RemoveContainer" containerID="1e44fd1264e16e1db549993ccea5312422456f76289f687fddfab04aaaefbdab" Jul 22 02:59:34.817770 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.817751 2557 scope.go:117] "RemoveContainer" containerID="39cc7c0c71ffe899064f1a305b4ad83ddffe6785502c76025157bfe7776bd0e9" Jul 22 02:59:34.818041 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:59:34.818022 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39cc7c0c71ffe899064f1a305b4ad83ddffe6785502c76025157bfe7776bd0e9\": container with ID starting with 39cc7c0c71ffe899064f1a305b4ad83ddffe6785502c76025157bfe7776bd0e9 not found: ID does not exist" containerID="39cc7c0c71ffe899064f1a305b4ad83ddffe6785502c76025157bfe7776bd0e9" Jul 22 02:59:34.818131 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.818055 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39cc7c0c71ffe899064f1a305b4ad83ddffe6785502c76025157bfe7776bd0e9"} err="failed to get container status \"39cc7c0c71ffe899064f1a305b4ad83ddffe6785502c76025157bfe7776bd0e9\": rpc error: code = NotFound desc = could not find container \"39cc7c0c71ffe899064f1a305b4ad83ddffe6785502c76025157bfe7776bd0e9\": container with ID starting with 39cc7c0c71ffe899064f1a305b4ad83ddffe6785502c76025157bfe7776bd0e9 not found: ID does not exist" Jul 22 02:59:34.818131 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.818078 2557 scope.go:117] "RemoveContainer" containerID="1e44fd1264e16e1db549993ccea5312422456f76289f687fddfab04aaaefbdab" Jul 22 02:59:34.818338 ip-10-0-132-119 kubenswrapper[2557]: E0722 02:59:34.818316 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e44fd1264e16e1db549993ccea5312422456f76289f687fddfab04aaaefbdab\": container with ID starting with 1e44fd1264e16e1db549993ccea5312422456f76289f687fddfab04aaaefbdab not found: ID does not exist" containerID="1e44fd1264e16e1db549993ccea5312422456f76289f687fddfab04aaaefbdab" Jul 22 02:59:34.818381 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.818347 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e44fd1264e16e1db549993ccea5312422456f76289f687fddfab04aaaefbdab"} err="failed to get container status \"1e44fd1264e16e1db549993ccea5312422456f76289f687fddfab04aaaefbdab\": rpc error: code = NotFound desc = could not find container \"1e44fd1264e16e1db549993ccea5312422456f76289f687fddfab04aaaefbdab\": container with ID starting with 1e44fd1264e16e1db549993ccea5312422456f76289f687fddfab04aaaefbdab not found: ID does not exist" Jul 22 02:59:34.825267 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.825242 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g"] Jul 22 02:59:34.828918 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:34.828899 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-3f8fef-openshift-default-c7dc96bbc-q9l2g"] Jul 22 02:59:35.609998 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:35.609965 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cf5a8bd-2475-4def-b4bc-364e413683dc" path="/var/lib/kubelet/pods/7cf5a8bd-2475-4def-b4bc-364e413683dc/volumes" Jul 22 02:59:57.947912 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:57.947813 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp"] Jul 22 02:59:57.948343 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:57.948031 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" podUID="1fe80d4e-f4fd-40b6-a6b0-090717bba83d" containerName="istio-proxy" containerID="cri-o://5b95126dde347d009e4067389795842f492e71c9b11cf26531a11848dbaef275" gracePeriod=30 Jul 22 02:59:58.926892 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:58.926850 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc"] Jul 22 02:59:58.927231 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:58.927218 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cf5a8bd-2475-4def-b4bc-364e413683dc" containerName="istio-proxy" Jul 22 02:59:58.927277 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:58.927233 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cf5a8bd-2475-4def-b4bc-364e413683dc" containerName="istio-proxy" Jul 22 02:59:58.927277 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:58.927248 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cf5a8bd-2475-4def-b4bc-364e413683dc" containerName="istio-proxy" Jul 22 02:59:58.927277 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:58.927255 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cf5a8bd-2475-4def-b4bc-364e413683dc" containerName="istio-proxy" Jul 22 02:59:58.927374 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:58.927319 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cf5a8bd-2475-4def-b4bc-364e413683dc" containerName="istio-proxy" Jul 22 02:59:58.927374 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:58.927326 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cf5a8bd-2475-4def-b4bc-364e413683dc" containerName="istio-proxy" Jul 22 02:59:58.929940 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:58.929923 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:58.937587 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:58.937557 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-995f8f-openshift-default-dockercfg-dxx7q\"" Jul 22 02:59:58.952977 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:58.952943 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc"] Jul 22 02:59:59.046879 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.046847 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-token\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.047083 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.046904 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-workload-socket\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.047083 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.046935 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-envoy\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.047083 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.046955 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-workload-certs\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.047083 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.046980 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-credential-socket\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.047083 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.047004 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6286c\" (UniqueName: \"kubernetes.io/projected/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-kube-api-access-6286c\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.047083 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.047039 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istiod-ca-cert\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.047397 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.047125 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-data\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.047397 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.047145 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-podinfo\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.148201 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.148158 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-data\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.148406 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.148218 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-podinfo\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.148406 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.148333 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-token\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.148532 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.148405 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-workload-socket\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.148532 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.148460 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-envoy\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.148532 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.148491 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-workload-certs\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.148686 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.148538 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-credential-socket\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.148686 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.148564 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6286c\" (UniqueName: \"kubernetes.io/projected/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-kube-api-access-6286c\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.148686 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.148626 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istiod-ca-cert\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.148686 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.148639 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-data\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.148938 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.148729 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-workload-socket\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.149001 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.148953 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-credential-socket\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.149067 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.149010 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-workload-certs\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.149344 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.149321 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istiod-ca-cert\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.150860 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.150814 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-envoy\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.151516 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.151489 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-podinfo\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.159325 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.159290 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-token\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.159586 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.159565 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6286c\" (UniqueName: \"kubernetes.io/projected/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-kube-api-access-6286c\") pod \"e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.240497 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.240460 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 02:59:59.398917 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.398880 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc"] Jul 22 02:59:59.400271 ip-10-0-132-119 kubenswrapper[2557]: W0722 02:59:59.400238 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fd3ecd3_e5fb_4ff8_9360_05a13b6716ea.slice/crio-3238eeaf188ad65360a66d4bab7d45bcb1948cbca3f816c0903d746ecc7bab10 WatchSource:0}: Error finding container 3238eeaf188ad65360a66d4bab7d45bcb1948cbca3f816c0903d746ecc7bab10: Status 404 returned error can't find the container with id 3238eeaf188ad65360a66d4bab7d45bcb1948cbca3f816c0903d746ecc7bab10 Jul 22 02:59:59.402591 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.402553 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:59:59.402682 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.402654 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:59:59.402722 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.402695 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:59:59.925480 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.925432 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" event={"ID":"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea","Type":"ContainerStarted","Data":"b5bd28379ea8339e22296d57e573dbfc4b2d12cd8233eb0734c69eab52b38977"} Jul 22 02:59:59.925659 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.925496 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" event={"ID":"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea","Type":"ContainerStarted","Data":"3238eeaf188ad65360a66d4bab7d45bcb1948cbca3f816c0903d746ecc7bab10"} Jul 22 02:59:59.950915 ip-10-0-132-119 kubenswrapper[2557]: I0722 02:59:59.950867 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" podStartSLOduration=1.950850484 podStartE2EDuration="1.950850484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:59:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:59:59.950021717 +0000 UTC m=+2430.858886725" watchObservedRunningTime="2026-07-22 02:59:59.950850484 +0000 UTC m=+2430.859715479" Jul 22 03:00:00.240670 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:00.240629 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 03:00:00.246250 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:00.246223 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 03:00:00.930989 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:00.930952 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 03:00:00.931811 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:00.931794 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 03:00:03.207035 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.206974 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 03:00:03.285626 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.285570 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-workload-socket\") pod \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " Jul 22 03:00:03.286455 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.285903 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-envoy\") pod \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " Jul 22 03:00:03.286455 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.286001 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-data\") pod \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " Jul 22 03:00:03.286455 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.286028 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-token\") pod \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " Jul 22 03:00:03.286455 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.286069 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-workload-certs\") pod \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " Jul 22 03:00:03.286455 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.286112 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-668c5\" (UniqueName: \"kubernetes.io/projected/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-kube-api-access-668c5\") pod \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " Jul 22 03:00:03.286455 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.286141 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-podinfo\") pod \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " Jul 22 03:00:03.286455 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.286171 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istiod-ca-cert\") pod \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " Jul 22 03:00:03.286455 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.286244 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-credential-socket\") pod \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\" (UID: \"1fe80d4e-f4fd-40b6-a6b0-090717bba83d\") " Jul 22 03:00:03.287013 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.286815 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1fe80d4e-f4fd-40b6-a6b0-090717bba83d" (UID: "1fe80d4e-f4fd-40b6-a6b0-090717bba83d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:00:03.287359 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.287041 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1fe80d4e-f4fd-40b6-a6b0-090717bba83d" (UID: "1fe80d4e-f4fd-40b6-a6b0-090717bba83d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:00:03.287359 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.287330 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1fe80d4e-f4fd-40b6-a6b0-090717bba83d" (UID: "1fe80d4e-f4fd-40b6-a6b0-090717bba83d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:00:03.288756 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.287565 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1fe80d4e-f4fd-40b6-a6b0-090717bba83d" (UID: "1fe80d4e-f4fd-40b6-a6b0-090717bba83d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:00:03.288756 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.288714 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-data" (OuterVolumeSpecName: "istio-data") pod "1fe80d4e-f4fd-40b6-a6b0-090717bba83d" (UID: "1fe80d4e-f4fd-40b6-a6b0-090717bba83d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:00:03.291460 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.291426 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1fe80d4e-f4fd-40b6-a6b0-090717bba83d" (UID: "1fe80d4e-f4fd-40b6-a6b0-090717bba83d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:00:03.291460 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.291454 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-kube-api-access-668c5" (OuterVolumeSpecName: "kube-api-access-668c5") pod "1fe80d4e-f4fd-40b6-a6b0-090717bba83d" (UID: "1fe80d4e-f4fd-40b6-a6b0-090717bba83d"). InnerVolumeSpecName "kube-api-access-668c5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:00:03.291748 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.291716 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1fe80d4e-f4fd-40b6-a6b0-090717bba83d" (UID: "1fe80d4e-f4fd-40b6-a6b0-090717bba83d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:00:03.291867 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.291777 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-token" (OuterVolumeSpecName: "istio-token") pod "1fe80d4e-f4fd-40b6-a6b0-090717bba83d" (UID: "1fe80d4e-f4fd-40b6-a6b0-090717bba83d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:00:03.388047 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.388001 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-workload-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:00:03.388047 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.388045 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-envoy\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:00:03.388302 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.388061 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-data\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:00:03.388302 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.388074 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-token\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:00:03.388302 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.388085 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-workload-certs\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:00:03.388302 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.388098 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-668c5\" (UniqueName: \"kubernetes.io/projected/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-kube-api-access-668c5\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:00:03.388302 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.388113 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istio-podinfo\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:00:03.388302 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.388126 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-istiod-ca-cert\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:00:03.388302 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.388139 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fe80d4e-f4fd-40b6-a6b0-090717bba83d-credential-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:00:03.947416 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.947311 2557 generic.go:356] "Generic (PLEG): container finished" podID="1fe80d4e-f4fd-40b6-a6b0-090717bba83d" containerID="5b95126dde347d009e4067389795842f492e71c9b11cf26531a11848dbaef275" exitCode=0 Jul 22 03:00:03.947416 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.947363 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" Jul 22 03:00:03.947611 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.947412 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" event={"ID":"1fe80d4e-f4fd-40b6-a6b0-090717bba83d","Type":"ContainerDied","Data":"5b95126dde347d009e4067389795842f492e71c9b11cf26531a11848dbaef275"} Jul 22 03:00:03.947611 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.947467 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp" event={"ID":"1fe80d4e-f4fd-40b6-a6b0-090717bba83d","Type":"ContainerDied","Data":"a49ee879dc39c7d1e58ce42003e2f24352d62101080a964b48945d2729a2f34b"} Jul 22 03:00:03.947611 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.947490 2557 scope.go:117] "RemoveContainer" containerID="5b95126dde347d009e4067389795842f492e71c9b11cf26531a11848dbaef275" Jul 22 03:00:03.955979 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.955961 2557 scope.go:117] "RemoveContainer" containerID="7f701715f1803ead853d11e874db8a6f139c772416b9f4b6db907893086db40c" Jul 22 03:00:03.965996 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.965973 2557 scope.go:117] "RemoveContainer" containerID="5b95126dde347d009e4067389795842f492e71c9b11cf26531a11848dbaef275" Jul 22 03:00:03.966251 ip-10-0-132-119 kubenswrapper[2557]: E0722 03:00:03.966230 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b95126dde347d009e4067389795842f492e71c9b11cf26531a11848dbaef275\": container with ID starting with 5b95126dde347d009e4067389795842f492e71c9b11cf26531a11848dbaef275 not found: ID does not exist" containerID="5b95126dde347d009e4067389795842f492e71c9b11cf26531a11848dbaef275" Jul 22 03:00:03.966305 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.966260 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b95126dde347d009e4067389795842f492e71c9b11cf26531a11848dbaef275"} err="failed to get container status \"5b95126dde347d009e4067389795842f492e71c9b11cf26531a11848dbaef275\": rpc error: code = NotFound desc = could not find container \"5b95126dde347d009e4067389795842f492e71c9b11cf26531a11848dbaef275\": container with ID starting with 5b95126dde347d009e4067389795842f492e71c9b11cf26531a11848dbaef275 not found: ID does not exist" Jul 22 03:00:03.966305 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.966276 2557 scope.go:117] "RemoveContainer" containerID="7f701715f1803ead853d11e874db8a6f139c772416b9f4b6db907893086db40c" Jul 22 03:00:03.966542 ip-10-0-132-119 kubenswrapper[2557]: E0722 03:00:03.966523 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f701715f1803ead853d11e874db8a6f139c772416b9f4b6db907893086db40c\": container with ID starting with 7f701715f1803ead853d11e874db8a6f139c772416b9f4b6db907893086db40c not found: ID does not exist" containerID="7f701715f1803ead853d11e874db8a6f139c772416b9f4b6db907893086db40c" Jul 22 03:00:03.966606 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.966551 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f701715f1803ead853d11e874db8a6f139c772416b9f4b6db907893086db40c"} err="failed to get container status \"7f701715f1803ead853d11e874db8a6f139c772416b9f4b6db907893086db40c\": rpc error: code = NotFound desc = could not find container \"7f701715f1803ead853d11e874db8a6f139c772416b9f4b6db907893086db40c\": container with ID starting with 7f701715f1803ead853d11e874db8a6f139c772416b9f4b6db907893086db40c not found: ID does not exist" Jul 22 03:00:03.972807 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.972783 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp"] Jul 22 03:00:03.977363 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:03.977343 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-dac71a-openshift-default-cb484f6b7-frpfp"] Jul 22 03:00:05.609634 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:05.609598 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fe80d4e-f4fd-40b6-a6b0-090717bba83d" path="/var/lib/kubelet/pods/1fe80d4e-f4fd-40b6-a6b0-090717bba83d/volumes" Jul 22 03:00:07.240238 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.240205 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9"] Jul 22 03:00:07.240613 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.240600 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fe80d4e-f4fd-40b6-a6b0-090717bba83d" containerName="istio-proxy" Jul 22 03:00:07.240613 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.240614 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fe80d4e-f4fd-40b6-a6b0-090717bba83d" containerName="istio-proxy" Jul 22 03:00:07.240707 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.240622 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fe80d4e-f4fd-40b6-a6b0-090717bba83d" containerName="istio-proxy" Jul 22 03:00:07.240707 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.240627 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fe80d4e-f4fd-40b6-a6b0-090717bba83d" containerName="istio-proxy" Jul 22 03:00:07.240770 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.240712 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fe80d4e-f4fd-40b6-a6b0-090717bba83d" containerName="istio-proxy" Jul 22 03:00:07.240770 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.240721 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fe80d4e-f4fd-40b6-a6b0-090717bba83d" containerName="istio-proxy" Jul 22 03:00:07.243783 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.243764 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.257438 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.257414 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-bb8c9d-openshift-default-dockercfg-xhb7z\"" Jul 22 03:00:07.273470 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.273446 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9"] Jul 22 03:00:07.426708 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.426673 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46lvk\" (UniqueName: \"kubernetes.io/projected/11fa53d6-304a-430b-923b-5b123ead5676-kube-api-access-46lvk\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.426920 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.426719 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11fa53d6-304a-430b-923b-5b123ead5676-istio-token\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.426920 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.426775 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11fa53d6-304a-430b-923b-5b123ead5676-istio-podinfo\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.426920 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.426814 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-workload-socket\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.426920 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.426848 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-credential-socket\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.426920 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.426874 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11fa53d6-304a-430b-923b-5b123ead5676-istiod-ca-cert\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.426920 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.426902 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-istio-envoy\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.427140 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.427040 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-istio-data\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.427140 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.427067 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-workload-certs\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.528446 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.528350 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-istio-envoy\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.528613 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.528451 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-istio-data\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.528613 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.528471 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-workload-certs\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.528613 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.528495 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46lvk\" (UniqueName: \"kubernetes.io/projected/11fa53d6-304a-430b-923b-5b123ead5676-kube-api-access-46lvk\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.528613 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.528527 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11fa53d6-304a-430b-923b-5b123ead5676-istio-token\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.528613 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.528558 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11fa53d6-304a-430b-923b-5b123ead5676-istio-podinfo\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.528613 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.528587 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-workload-socket\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.528613 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.528611 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-credential-socket\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.529017 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.528777 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11fa53d6-304a-430b-923b-5b123ead5676-istiod-ca-cert\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.529017 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.528907 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-workload-certs\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.529017 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.528942 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-istio-data\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.529176 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.529127 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-credential-socket\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.529221 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.529191 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-workload-socket\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.529406 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.529388 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11fa53d6-304a-430b-923b-5b123ead5676-istiod-ca-cert\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.531285 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.531261 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-istio-envoy\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.531692 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.531630 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11fa53d6-304a-430b-923b-5b123ead5676-istio-podinfo\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.541133 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.541109 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46lvk\" (UniqueName: \"kubernetes.io/projected/11fa53d6-304a-430b-923b-5b123ead5676-kube-api-access-46lvk\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.544017 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.543996 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11fa53d6-304a-430b-923b-5b123ead5676-istio-token\") pod \"e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.556365 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.556344 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:07.715642 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.715613 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9"] Jul 22 03:00:07.717424 ip-10-0-132-119 kubenswrapper[2557]: W0722 03:00:07.717391 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11fa53d6_304a_430b_923b_5b123ead5676.slice/crio-c2631a7ef692b7ddde65928fcf73bde12f869b270e26934446da47c172de7103 WatchSource:0}: Error finding container c2631a7ef692b7ddde65928fcf73bde12f869b270e26934446da47c172de7103: Status 404 returned error can't find the container with id c2631a7ef692b7ddde65928fcf73bde12f869b270e26934446da47c172de7103 Jul 22 03:00:07.719503 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.719469 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 03:00:07.719599 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.719552 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 03:00:07.719667 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.719595 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 03:00:07.970187 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.970151 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" event={"ID":"11fa53d6-304a-430b-923b-5b123ead5676","Type":"ContainerStarted","Data":"23476d706fafed2166ab0c24461b443a2f7adefc5457c8c5cde2123f8f2aed07"} Jul 22 03:00:07.970350 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:07.970215 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" event={"ID":"11fa53d6-304a-430b-923b-5b123ead5676","Type":"ContainerStarted","Data":"c2631a7ef692b7ddde65928fcf73bde12f869b270e26934446da47c172de7103"} Jul 22 03:00:08.557556 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:08.557521 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:08.562269 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:08.562245 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:08.597563 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:08.597507 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" podStartSLOduration=1.597488051 podStartE2EDuration="1.597488051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:00:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:00:08.017972647 +0000 UTC m=+2438.926837644" watchObservedRunningTime="2026-07-22 03:00:08.597488051 +0000 UTC m=+2439.506353047" Jul 22 03:00:08.975228 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:08.975198 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:08.976265 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:08.976243 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:16.143973 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:16.143848 2557 patch_prober.go:28] interesting pod/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": read tcp 10.133.0.2:52774->10.133.0.52:15021: read: connection reset by peer" start-of-body= Jul 22 03:00:16.143973 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:16.143910 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" podUID="3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": read tcp 10.133.0.2:52774->10.133.0.52:15021: read: connection reset by peer" Jul 22 03:00:17.023341 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:17.023306 2557 generic.go:356] "Generic (PLEG): container finished" podID="3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" containerID="b5bd28379ea8339e22296d57e573dbfc4b2d12cd8233eb0734c69eab52b38977" exitCode=0 Jul 22 03:00:17.023521 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:17.023352 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" event={"ID":"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea","Type":"ContainerDied","Data":"b5bd28379ea8339e22296d57e573dbfc4b2d12cd8233eb0734c69eab52b38977"} Jul 22 03:00:17.023720 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:17.023707 2557 scope.go:117] "RemoveContainer" containerID="b5bd28379ea8339e22296d57e573dbfc4b2d12cd8233eb0734c69eab52b38977" Jul 22 03:00:17.024758 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:17.024731 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 03:00:17.024814 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:17.024796 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 03:00:17.024888 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:17.024848 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 03:00:18.029788 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:18.029755 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" event={"ID":"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea","Type":"ContainerStarted","Data":"c8c6e0e4fc49644a2dd47f3a4dd032cb42cd4ac0bfa7de82be94fe8c861ea3a4"} Jul 22 03:00:18.240964 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:18.240928 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 03:00:18.241256 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:18.241232 2557 patch_prober.go:28] interesting pod/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 22 03:00:18.241355 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:18.241290 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" podUID="3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 22 03:00:19.241312 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:19.241276 2557 patch_prober.go:28] interesting pod/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 22 03:00:19.241746 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:19.241346 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" podUID="3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 22 03:00:20.240932 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:20.240878 2557 patch_prober.go:28] interesting pod/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 22 03:00:20.241170 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:20.240971 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" podUID="3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 22 03:00:21.241175 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:21.241137 2557 patch_prober.go:28] interesting pod/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 22 03:00:21.241564 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:21.241189 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" podUID="3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 22 03:00:22.244366 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:22.244336 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 03:00:22.244806 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:22.244786 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 03:00:22.245521 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:22.245498 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 03:00:25.017069 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:25.017012 2557 patch_prober.go:28] interesting pod/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 03:00:25.017509 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:25.017104 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" podUID="11fa53d6-304a-430b-923b-5b123ead5676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 03:00:27.077143 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:27.077109 2557 generic.go:356] "Generic (PLEG): container finished" podID="11fa53d6-304a-430b-923b-5b123ead5676" containerID="23476d706fafed2166ab0c24461b443a2f7adefc5457c8c5cde2123f8f2aed07" exitCode=0 Jul 22 03:00:27.077576 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:27.077171 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" event={"ID":"11fa53d6-304a-430b-923b-5b123ead5676","Type":"ContainerDied","Data":"23476d706fafed2166ab0c24461b443a2f7adefc5457c8c5cde2123f8f2aed07"} Jul 22 03:00:27.077576 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:27.077523 2557 scope.go:117] "RemoveContainer" containerID="23476d706fafed2166ab0c24461b443a2f7adefc5457c8c5cde2123f8f2aed07" Jul 22 03:00:27.078511 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:27.078483 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 03:00:27.078608 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:27.078548 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 03:00:27.078608 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:27.078577 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 03:00:28.102848 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:28.098957 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" event={"ID":"11fa53d6-304a-430b-923b-5b123ead5676","Type":"ContainerStarted","Data":"36a0f2e3b51448fae26f54c79f175b1a6eca7e3f5ad952049d0ee239c151a03a"} Jul 22 03:00:28.556758 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:28.556726 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:28.557065 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:28.557040 2557 patch_prober.go:28] interesting pod/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 22 03:00:28.557167 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:28.557096 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" podUID="11fa53d6-304a-430b-923b-5b123ead5676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 22 03:00:29.556942 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:29.556903 2557 patch_prober.go:28] interesting pod/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 22 03:00:29.557347 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:29.556961 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" podUID="11fa53d6-304a-430b-923b-5b123ead5676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 22 03:00:30.557432 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:30.557398 2557 patch_prober.go:28] interesting pod/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 22 03:00:30.557818 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:30.557467 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" podUID="11fa53d6-304a-430b-923b-5b123ead5676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 22 03:00:31.557568 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:31.557533 2557 patch_prober.go:28] interesting pod/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 22 03:00:31.558023 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:31.557593 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" podUID="11fa53d6-304a-430b-923b-5b123ead5676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 22 03:00:32.560700 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:32.560670 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:32.561140 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:32.561046 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:00:32.561646 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:00:32.561627 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:02:29.862851 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:02:29.862804 2557 scope.go:117] "RemoveContainer" containerID="6d0006f08198b44f07bfc40031860f230a85f1a704ba4eb6d0e9dfa8d4db816b" Jul 22 03:06:52.222369 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:52.222333 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc"] Jul 22 03:06:52.222910 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:52.222561 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" podUID="3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" containerName="istio-proxy" containerID="cri-o://c8c6e0e4fc49644a2dd47f3a4dd032cb42cd4ac0bfa7de82be94fe8c861ea3a4" gracePeriod=30 Jul 22 03:06:52.246863 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:52.246798 2557 patch_prober.go:28] interesting pod/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 03:06:52.247026 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:52.246890 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" podUID="3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 03:06:57.485464 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.485406 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 03:06:57.650864 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.650798 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-workload-socket\") pod \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " Jul 22 03:06:57.651077 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.650941 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-envoy\") pod \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " Jul 22 03:06:57.651077 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.650970 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6286c\" (UniqueName: \"kubernetes.io/projected/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-kube-api-access-6286c\") pod \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " Jul 22 03:06:57.651077 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.651012 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-token\") pod \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " Jul 22 03:06:57.651077 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.651034 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-workload-certs\") pod \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " Jul 22 03:06:57.651077 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.651060 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-credential-socket\") pod \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " Jul 22 03:06:57.651337 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.651066 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" (UID: "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:06:57.651337 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.651106 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istiod-ca-cert\") pod \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " Jul 22 03:06:57.651337 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.651139 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-podinfo\") pod \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " Jul 22 03:06:57.651337 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.651181 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-data\") pod \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\" (UID: \"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea\") " Jul 22 03:06:57.651526 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.651459 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" (UID: "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:06:57.651526 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.651506 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-workload-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:06:57.651634 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.651563 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" (UID: "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:06:57.651949 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.651927 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" (UID: "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:06:57.652473 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.652450 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-data" (OuterVolumeSpecName: "istio-data") pod "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" (UID: "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:06:57.653377 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.653352 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" (UID: "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:06:57.653500 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.653482 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" (UID: "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:06:57.653782 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.653756 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-token" (OuterVolumeSpecName: "istio-token") pod "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" (UID: "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:06:57.653889 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.653756 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-kube-api-access-6286c" (OuterVolumeSpecName: "kube-api-access-6286c") pod "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" (UID: "3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea"). InnerVolumeSpecName "kube-api-access-6286c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:06:57.752860 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.752748 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-envoy\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:06:57.752860 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.752773 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6286c\" (UniqueName: \"kubernetes.io/projected/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-kube-api-access-6286c\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:06:57.752860 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.752782 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-token\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:06:57.752860 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.752791 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-workload-certs\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:06:57.752860 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.752798 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-credential-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:06:57.752860 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.752807 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istiod-ca-cert\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:06:57.752860 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.752816 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-podinfo\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:06:57.752860 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.752853 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea-istio-data\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:06:57.996875 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.996816 2557 generic.go:356] "Generic (PLEG): container finished" podID="3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" containerID="c8c6e0e4fc49644a2dd47f3a4dd032cb42cd4ac0bfa7de82be94fe8c861ea3a4" exitCode=0 Jul 22 03:06:57.997060 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.996898 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" Jul 22 03:06:57.997060 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.996931 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" event={"ID":"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea","Type":"ContainerDied","Data":"c8c6e0e4fc49644a2dd47f3a4dd032cb42cd4ac0bfa7de82be94fe8c861ea3a4"} Jul 22 03:06:57.997060 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.996998 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc" event={"ID":"3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea","Type":"ContainerDied","Data":"3238eeaf188ad65360a66d4bab7d45bcb1948cbca3f816c0903d746ecc7bab10"} Jul 22 03:06:57.997060 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:57.997036 2557 scope.go:117] "RemoveContainer" containerID="c8c6e0e4fc49644a2dd47f3a4dd032cb42cd4ac0bfa7de82be94fe8c861ea3a4" Jul 22 03:06:58.010961 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:58.010935 2557 scope.go:117] "RemoveContainer" containerID="b5bd28379ea8339e22296d57e573dbfc4b2d12cd8233eb0734c69eab52b38977" Jul 22 03:06:58.019387 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:58.019367 2557 scope.go:117] "RemoveContainer" containerID="c8c6e0e4fc49644a2dd47f3a4dd032cb42cd4ac0bfa7de82be94fe8c861ea3a4" Jul 22 03:06:58.019628 ip-10-0-132-119 kubenswrapper[2557]: E0722 03:06:58.019611 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8c6e0e4fc49644a2dd47f3a4dd032cb42cd4ac0bfa7de82be94fe8c861ea3a4\": container with ID starting with c8c6e0e4fc49644a2dd47f3a4dd032cb42cd4ac0bfa7de82be94fe8c861ea3a4 not found: ID does not exist" containerID="c8c6e0e4fc49644a2dd47f3a4dd032cb42cd4ac0bfa7de82be94fe8c861ea3a4" Jul 22 03:06:58.019678 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:58.019636 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8c6e0e4fc49644a2dd47f3a4dd032cb42cd4ac0bfa7de82be94fe8c861ea3a4"} err="failed to get container status \"c8c6e0e4fc49644a2dd47f3a4dd032cb42cd4ac0bfa7de82be94fe8c861ea3a4\": rpc error: code = NotFound desc = could not find container \"c8c6e0e4fc49644a2dd47f3a4dd032cb42cd4ac0bfa7de82be94fe8c861ea3a4\": container with ID starting with c8c6e0e4fc49644a2dd47f3a4dd032cb42cd4ac0bfa7de82be94fe8c861ea3a4 not found: ID does not exist" Jul 22 03:06:58.019678 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:58.019652 2557 scope.go:117] "RemoveContainer" containerID="b5bd28379ea8339e22296d57e573dbfc4b2d12cd8233eb0734c69eab52b38977" Jul 22 03:06:58.019889 ip-10-0-132-119 kubenswrapper[2557]: E0722 03:06:58.019871 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5bd28379ea8339e22296d57e573dbfc4b2d12cd8233eb0734c69eab52b38977\": container with ID starting with b5bd28379ea8339e22296d57e573dbfc4b2d12cd8233eb0734c69eab52b38977 not found: ID does not exist" containerID="b5bd28379ea8339e22296d57e573dbfc4b2d12cd8233eb0734c69eab52b38977" Jul 22 03:06:58.019943 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:58.019899 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5bd28379ea8339e22296d57e573dbfc4b2d12cd8233eb0734c69eab52b38977"} err="failed to get container status \"b5bd28379ea8339e22296d57e573dbfc4b2d12cd8233eb0734c69eab52b38977\": rpc error: code = NotFound desc = could not find container \"b5bd28379ea8339e22296d57e573dbfc4b2d12cd8233eb0734c69eab52b38977\": container with ID starting with b5bd28379ea8339e22296d57e573dbfc4b2d12cd8233eb0734c69eab52b38977 not found: ID does not exist" Jul 22 03:06:58.039695 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:58.039668 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc"] Jul 22 03:06:58.043801 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:58.043780 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-995f8f-openshift-default-59f7b6b8f5-4nmgc"] Jul 22 03:06:59.608655 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:06:59.608615 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" path="/var/lib/kubelet/pods/3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea/volumes" Jul 22 03:07:14.731390 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:14.731348 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9"] Jul 22 03:07:14.731922 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:14.731572 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" podUID="11fa53d6-304a-430b-923b-5b123ead5676" containerName="istio-proxy" containerID="cri-o://36a0f2e3b51448fae26f54c79f175b1a6eca7e3f5ad952049d0ee239c151a03a" gracePeriod=30 Jul 22 03:07:15.410204 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:15.410166 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dc4cf4958-8f6qx_bb5c5554-0112-4399-ac9f-bb3f421258e9/authorino/0.log" Jul 22 03:07:17.563326 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:17.563289 2557 patch_prober.go:28] interesting pod/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 03:07:17.563716 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:17.563384 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" podUID="11fa53d6-304a-430b-923b-5b123ead5676" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 03:07:19.979953 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:19.979924 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:07:20.026570 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.026481 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-workload-socket\") pod \"11fa53d6-304a-430b-923b-5b123ead5676\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " Jul 22 03:07:20.026570 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.026519 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11fa53d6-304a-430b-923b-5b123ead5676-istio-podinfo\") pod \"11fa53d6-304a-430b-923b-5b123ead5676\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " Jul 22 03:07:20.026570 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.026540 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-istio-envoy\") pod \"11fa53d6-304a-430b-923b-5b123ead5676\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " Jul 22 03:07:20.026570 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.026570 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-credential-socket\") pod \"11fa53d6-304a-430b-923b-5b123ead5676\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " Jul 22 03:07:20.026946 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.026603 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46lvk\" (UniqueName: \"kubernetes.io/projected/11fa53d6-304a-430b-923b-5b123ead5676-kube-api-access-46lvk\") pod \"11fa53d6-304a-430b-923b-5b123ead5676\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " Jul 22 03:07:20.026946 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.026630 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-istio-data\") pod \"11fa53d6-304a-430b-923b-5b123ead5676\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " Jul 22 03:07:20.026946 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.026648 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11fa53d6-304a-430b-923b-5b123ead5676-istio-token\") pod \"11fa53d6-304a-430b-923b-5b123ead5676\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " Jul 22 03:07:20.026946 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.026877 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "11fa53d6-304a-430b-923b-5b123ead5676" (UID: "11fa53d6-304a-430b-923b-5b123ead5676"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:07:20.026946 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.026922 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11fa53d6-304a-430b-923b-5b123ead5676-istiod-ca-cert\") pod \"11fa53d6-304a-430b-923b-5b123ead5676\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " Jul 22 03:07:20.027227 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.026944 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "11fa53d6-304a-430b-923b-5b123ead5676" (UID: "11fa53d6-304a-430b-923b-5b123ead5676"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:07:20.027227 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.026991 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-workload-certs\") pod \"11fa53d6-304a-430b-923b-5b123ead5676\" (UID: \"11fa53d6-304a-430b-923b-5b123ead5676\") " Jul 22 03:07:20.027227 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.027209 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11fa53d6-304a-430b-923b-5b123ead5676-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "11fa53d6-304a-430b-923b-5b123ead5676" (UID: "11fa53d6-304a-430b-923b-5b123ead5676"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:07:20.027983 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.027457 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11fa53d6-304a-430b-923b-5b123ead5676-istiod-ca-cert\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:07:20.027983 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.027481 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-workload-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:07:20.027983 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.027496 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-credential-socket\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:07:20.027983 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.027528 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "11fa53d6-304a-430b-923b-5b123ead5676" (UID: "11fa53d6-304a-430b-923b-5b123ead5676"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:07:20.028283 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.028186 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-istio-data" (OuterVolumeSpecName: "istio-data") pod "11fa53d6-304a-430b-923b-5b123ead5676" (UID: "11fa53d6-304a-430b-923b-5b123ead5676"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:07:20.028887 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.028859 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "11fa53d6-304a-430b-923b-5b123ead5676" (UID: "11fa53d6-304a-430b-923b-5b123ead5676"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:07:20.029341 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.029316 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11fa53d6-304a-430b-923b-5b123ead5676-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "11fa53d6-304a-430b-923b-5b123ead5676" (UID: "11fa53d6-304a-430b-923b-5b123ead5676"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 03:07:20.029430 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.029369 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11fa53d6-304a-430b-923b-5b123ead5676-kube-api-access-46lvk" (OuterVolumeSpecName: "kube-api-access-46lvk") pod "11fa53d6-304a-430b-923b-5b123ead5676" (UID: "11fa53d6-304a-430b-923b-5b123ead5676"). InnerVolumeSpecName "kube-api-access-46lvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:07:20.029430 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.029390 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11fa53d6-304a-430b-923b-5b123ead5676-istio-token" (OuterVolumeSpecName: "istio-token") pod "11fa53d6-304a-430b-923b-5b123ead5676" (UID: "11fa53d6-304a-430b-923b-5b123ead5676"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:07:20.106121 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.106090 2557 generic.go:356] "Generic (PLEG): container finished" podID="11fa53d6-304a-430b-923b-5b123ead5676" containerID="36a0f2e3b51448fae26f54c79f175b1a6eca7e3f5ad952049d0ee239c151a03a" exitCode=0 Jul 22 03:07:20.106313 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.106148 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" Jul 22 03:07:20.106313 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.106164 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" event={"ID":"11fa53d6-304a-430b-923b-5b123ead5676","Type":"ContainerDied","Data":"36a0f2e3b51448fae26f54c79f175b1a6eca7e3f5ad952049d0ee239c151a03a"} Jul 22 03:07:20.106313 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.106212 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9" event={"ID":"11fa53d6-304a-430b-923b-5b123ead5676","Type":"ContainerDied","Data":"c2631a7ef692b7ddde65928fcf73bde12f869b270e26934446da47c172de7103"} Jul 22 03:07:20.106313 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.106234 2557 scope.go:117] "RemoveContainer" containerID="36a0f2e3b51448fae26f54c79f175b1a6eca7e3f5ad952049d0ee239c151a03a" Jul 22 03:07:20.116418 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.116402 2557 scope.go:117] "RemoveContainer" containerID="23476d706fafed2166ab0c24461b443a2f7adefc5457c8c5cde2123f8f2aed07" Jul 22 03:07:20.126288 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.126256 2557 scope.go:117] "RemoveContainer" containerID="36a0f2e3b51448fae26f54c79f175b1a6eca7e3f5ad952049d0ee239c151a03a" Jul 22 03:07:20.126733 ip-10-0-132-119 kubenswrapper[2557]: E0722 03:07:20.126715 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36a0f2e3b51448fae26f54c79f175b1a6eca7e3f5ad952049d0ee239c151a03a\": container with ID starting with 36a0f2e3b51448fae26f54c79f175b1a6eca7e3f5ad952049d0ee239c151a03a not found: ID does not exist" containerID="36a0f2e3b51448fae26f54c79f175b1a6eca7e3f5ad952049d0ee239c151a03a" Jul 22 03:07:20.126787 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.126744 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36a0f2e3b51448fae26f54c79f175b1a6eca7e3f5ad952049d0ee239c151a03a"} err="failed to get container status \"36a0f2e3b51448fae26f54c79f175b1a6eca7e3f5ad952049d0ee239c151a03a\": rpc error: code = NotFound desc = could not find container \"36a0f2e3b51448fae26f54c79f175b1a6eca7e3f5ad952049d0ee239c151a03a\": container with ID starting with 36a0f2e3b51448fae26f54c79f175b1a6eca7e3f5ad952049d0ee239c151a03a not found: ID does not exist" Jul 22 03:07:20.126787 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.126762 2557 scope.go:117] "RemoveContainer" containerID="23476d706fafed2166ab0c24461b443a2f7adefc5457c8c5cde2123f8f2aed07" Jul 22 03:07:20.127031 ip-10-0-132-119 kubenswrapper[2557]: E0722 03:07:20.127014 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23476d706fafed2166ab0c24461b443a2f7adefc5457c8c5cde2123f8f2aed07\": container with ID starting with 23476d706fafed2166ab0c24461b443a2f7adefc5457c8c5cde2123f8f2aed07 not found: ID does not exist" containerID="23476d706fafed2166ab0c24461b443a2f7adefc5457c8c5cde2123f8f2aed07" Jul 22 03:07:20.127091 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.127037 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23476d706fafed2166ab0c24461b443a2f7adefc5457c8c5cde2123f8f2aed07"} err="failed to get container status \"23476d706fafed2166ab0c24461b443a2f7adefc5457c8c5cde2123f8f2aed07\": rpc error: code = NotFound desc = could not find container \"23476d706fafed2166ab0c24461b443a2f7adefc5457c8c5cde2123f8f2aed07\": container with ID starting with 23476d706fafed2166ab0c24461b443a2f7adefc5457c8c5cde2123f8f2aed07 not found: ID does not exist" Jul 22 03:07:20.128673 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.128648 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46lvk\" (UniqueName: \"kubernetes.io/projected/11fa53d6-304a-430b-923b-5b123ead5676-kube-api-access-46lvk\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:07:20.128759 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.128680 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-istio-data\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:07:20.128759 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.128694 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11fa53d6-304a-430b-923b-5b123ead5676-istio-token\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:07:20.128759 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.128706 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-workload-certs\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:07:20.128759 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.128718 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11fa53d6-304a-430b-923b-5b123ead5676-istio-podinfo\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:07:20.128759 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.128729 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11fa53d6-304a-430b-923b-5b123ead5676-istio-envoy\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Jul 22 03:07:20.130363 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.130341 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9"] Jul 22 03:07:20.134731 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.134709 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-bb8c9d-openshift-default-8446c49bf5-9vzb9"] Jul 22 03:07:20.833329 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.833250 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-57frf_5cd5e93b-ab6c-464b-a914-02ee4d72f695/manager/0.log" Jul 22 03:07:20.957479 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:20.957446 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-74cxr_32de991c-21b2-4846-94b6-52923946c2d4/manager/0.log" Jul 22 03:07:21.221944 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:21.221913 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-bqxlc_ef9503af-c09f-4cfc-a58d-b459e17ff907/server/0.log" Jul 22 03:07:21.337344 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:21.337304 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-wfh7b_42c5d187-3867-4a46-8f7c-980f572dc26a/manager/0.log" Jul 22 03:07:21.608851 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:21.608793 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11fa53d6-304a-430b-923b-5b123ead5676" path="/var/lib/kubelet/pods/11fa53d6-304a-430b-923b-5b123ead5676/volumes" Jul 22 03:07:21.714469 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:21.714443 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb9fd785c-xr8f8_e8e68259-7a67-470c-b249-733ac647e978/copy-manifests/0.log" Jul 22 03:07:21.721837 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:21.721799 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb9fd785c-xr8f8_e8e68259-7a67-470c-b249-733ac647e978/manager/0.log" Jul 22 03:07:23.741128 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:23.741100 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dc4cf4958-8f6qx_bb5c5554-0112-4399-ac9f-bb3f421258e9/authorino/0.log" Jul 22 03:07:23.858395 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:23.858362 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-jxqwm_b888f23c-adc1-46d7-91fb-53c1a676e036/manager/0.log" Jul 22 03:07:23.985802 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:23.985769 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-7cl6l_beac8017-6dbc-4668-83a9-47134c978970/manager/0.log" Jul 22 03:07:24.219315 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:24.219290 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-npftk_4cbddcc4-1c96-4012-a13d-b6859697723a/registry-server/0.log" Jul 22 03:07:24.339314 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:24.339281 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tb22g_d2094f78-fc10-4eb6-baea-24169f156c80/manager/0.log" Jul 22 03:07:24.613992 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:24.613902 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-fwz5b_9db3f385-a143-4024-be8a-9d10507eff8a/manager/0.log" Jul 22 03:07:24.967065 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:24.967042 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nnp5f_218745d1-9832-4760-b46f-3661fa7f4e5a/istio-proxy/0.log" Jul 22 03:07:25.319666 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:25.319593 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c4bb8965-gt6rw_b35918df-d4bf-4130-8e89-9a9b9ca4eb95/kube-auth-proxy/0.log" Jul 22 03:07:25.438499 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:25.438473 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hlntg_42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6/istio-proxy/1.log" Jul 22 03:07:26.138550 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:26.138523 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-p9jz5_078c49d1-ecca-46d4-81dd-4071bb34a763/main/0.log" Jul 22 03:07:26.390054 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:26.389976 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-x9q7p_2ff43abe-5ba8-4077-9907-9a1aab83aebb/main/0.log" Jul 22 03:07:35.305032 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:35.305002 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-w4bgx_ede3658a-126b-446f-b57a-678aefbc7936/global-pull-secret-syncer/0.log" Jul 22 03:07:35.399719 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:35.399663 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-b2b7n_e64b46d5-3ac5-43fd-8b38-253f8a8b04ad/konnectivity-agent/0.log" Jul 22 03:07:35.449306 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:35.449276 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-119.ec2.internal_11b830b0e1c88ebf098f99a124478494/haproxy/0.log" Jul 22 03:07:40.152408 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:40.152377 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dc4cf4958-8f6qx_bb5c5554-0112-4399-ac9f-bb3f421258e9/authorino/0.log" Jul 22 03:07:40.199437 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:40.199407 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-jxqwm_b888f23c-adc1-46d7-91fb-53c1a676e036/manager/0.log" Jul 22 03:07:40.220817 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:40.220788 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-7cl6l_beac8017-6dbc-4668-83a9-47134c978970/manager/0.log" Jul 22 03:07:40.294574 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:40.294540 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-npftk_4cbddcc4-1c96-4012-a13d-b6859697723a/registry-server/0.log" Jul 22 03:07:40.436014 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:40.435925 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tb22g_d2094f78-fc10-4eb6-baea-24169f156c80/manager/0.log" Jul 22 03:07:40.530721 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:40.530676 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-fwz5b_9db3f385-a143-4024-be8a-9d10507eff8a/manager/0.log" Jul 22 03:07:42.741803 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:42.741779 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qznvs_e4416035-20a9-41e2-a938-9dbaf8c68f20/node-exporter/0.log" Jul 22 03:07:42.766245 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:42.766211 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qznvs_e4416035-20a9-41e2-a938-9dbaf8c68f20/kube-rbac-proxy/0.log" Jul 22 03:07:42.790794 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:42.790772 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qznvs_e4416035-20a9-41e2-a938-9dbaf8c68f20/init-textfile/0.log" Jul 22 03:07:44.847010 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:44.846979 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-xsc6x_28e31c18-494c-4fd8-896d-b302c92de0a6/networking-console-plugin/0.log" Jul 22 03:07:45.774455 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.774416 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg"] Jul 22 03:07:45.774804 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.774791 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" containerName="istio-proxy" Jul 22 03:07:45.774908 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.774806 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" containerName="istio-proxy" Jul 22 03:07:45.774908 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.774833 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11fa53d6-304a-430b-923b-5b123ead5676" containerName="istio-proxy" Jul 22 03:07:45.774908 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.774840 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="11fa53d6-304a-430b-923b-5b123ead5676" containerName="istio-proxy" Jul 22 03:07:45.774908 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.774856 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11fa53d6-304a-430b-923b-5b123ead5676" containerName="istio-proxy" Jul 22 03:07:45.774908 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.774865 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="11fa53d6-304a-430b-923b-5b123ead5676" containerName="istio-proxy" Jul 22 03:07:45.775070 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.774926 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="11fa53d6-304a-430b-923b-5b123ead5676" containerName="istio-proxy" Jul 22 03:07:45.775070 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.774934 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" containerName="istio-proxy" Jul 22 03:07:45.775070 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.774941 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="11fa53d6-304a-430b-923b-5b123ead5676" containerName="istio-proxy" Jul 22 03:07:45.775070 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.774947 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fd3ecd3-e5fb-4ff8-9360-05a13b6716ea" containerName="istio-proxy" Jul 22 03:07:45.777946 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.777927 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:45.780666 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.780646 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c69w5\"/\"openshift-service-ca.crt\"" Jul 22 03:07:45.780755 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.780682 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-c69w5\"/\"default-dockercfg-hkft4\"" Jul 22 03:07:45.780809 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.780756 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c69w5\"/\"kube-root-ca.crt\"" Jul 22 03:07:45.785542 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.785522 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg"] Jul 22 03:07:45.850531 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.850489 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj2jd\" (UniqueName: \"kubernetes.io/projected/c687d6f1-4272-4aa7-a402-37cf213b1bf7-kube-api-access-hj2jd\") pod \"perf-node-gather-daemonset-6q9cg\" (UID: \"c687d6f1-4272-4aa7-a402-37cf213b1bf7\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:45.850912 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.850536 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c687d6f1-4272-4aa7-a402-37cf213b1bf7-lib-modules\") pod \"perf-node-gather-daemonset-6q9cg\" (UID: \"c687d6f1-4272-4aa7-a402-37cf213b1bf7\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:45.850912 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.850621 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c687d6f1-4272-4aa7-a402-37cf213b1bf7-proc\") pod \"perf-node-gather-daemonset-6q9cg\" (UID: \"c687d6f1-4272-4aa7-a402-37cf213b1bf7\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:45.850912 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.850649 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c687d6f1-4272-4aa7-a402-37cf213b1bf7-podres\") pod \"perf-node-gather-daemonset-6q9cg\" (UID: \"c687d6f1-4272-4aa7-a402-37cf213b1bf7\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:45.850912 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.850671 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c687d6f1-4272-4aa7-a402-37cf213b1bf7-sys\") pod \"perf-node-gather-daemonset-6q9cg\" (UID: \"c687d6f1-4272-4aa7-a402-37cf213b1bf7\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:45.951946 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.951906 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c687d6f1-4272-4aa7-a402-37cf213b1bf7-proc\") pod \"perf-node-gather-daemonset-6q9cg\" (UID: \"c687d6f1-4272-4aa7-a402-37cf213b1bf7\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:45.951946 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.951948 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c687d6f1-4272-4aa7-a402-37cf213b1bf7-podres\") pod \"perf-node-gather-daemonset-6q9cg\" (UID: \"c687d6f1-4272-4aa7-a402-37cf213b1bf7\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:45.952169 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.951975 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c687d6f1-4272-4aa7-a402-37cf213b1bf7-sys\") pod \"perf-node-gather-daemonset-6q9cg\" (UID: \"c687d6f1-4272-4aa7-a402-37cf213b1bf7\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:45.952169 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.952035 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c687d6f1-4272-4aa7-a402-37cf213b1bf7-proc\") pod \"perf-node-gather-daemonset-6q9cg\" (UID: \"c687d6f1-4272-4aa7-a402-37cf213b1bf7\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:45.952169 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.952063 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj2jd\" (UniqueName: \"kubernetes.io/projected/c687d6f1-4272-4aa7-a402-37cf213b1bf7-kube-api-access-hj2jd\") pod \"perf-node-gather-daemonset-6q9cg\" (UID: \"c687d6f1-4272-4aa7-a402-37cf213b1bf7\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:45.952169 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.952076 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c687d6f1-4272-4aa7-a402-37cf213b1bf7-podres\") pod \"perf-node-gather-daemonset-6q9cg\" (UID: \"c687d6f1-4272-4aa7-a402-37cf213b1bf7\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:45.952169 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.952111 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c687d6f1-4272-4aa7-a402-37cf213b1bf7-lib-modules\") pod \"perf-node-gather-daemonset-6q9cg\" (UID: \"c687d6f1-4272-4aa7-a402-37cf213b1bf7\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:45.952169 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.952127 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c687d6f1-4272-4aa7-a402-37cf213b1bf7-sys\") pod \"perf-node-gather-daemonset-6q9cg\" (UID: \"c687d6f1-4272-4aa7-a402-37cf213b1bf7\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:45.952371 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.952254 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c687d6f1-4272-4aa7-a402-37cf213b1bf7-lib-modules\") pod \"perf-node-gather-daemonset-6q9cg\" (UID: \"c687d6f1-4272-4aa7-a402-37cf213b1bf7\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:45.960752 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:45.960732 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj2jd\" (UniqueName: \"kubernetes.io/projected/c687d6f1-4272-4aa7-a402-37cf213b1bf7-kube-api-access-hj2jd\") pod \"perf-node-gather-daemonset-6q9cg\" (UID: \"c687d6f1-4272-4aa7-a402-37cf213b1bf7\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:46.088672 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:46.088580 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:46.226136 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:46.226109 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg"] Jul 22 03:07:46.227922 ip-10-0-132-119 kubenswrapper[2557]: W0722 03:07:46.227895 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc687d6f1_4272_4aa7_a402_37cf213b1bf7.slice/crio-f80f5ccd64a66f43dc8a7caa15eb659248c66cfd038e5baf7ed2fce446a6114b WatchSource:0}: Error finding container f80f5ccd64a66f43dc8a7caa15eb659248c66cfd038e5baf7ed2fce446a6114b: Status 404 returned error can't find the container with id f80f5ccd64a66f43dc8a7caa15eb659248c66cfd038e5baf7ed2fce446a6114b Jul 22 03:07:47.233610 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:47.233567 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" event={"ID":"c687d6f1-4272-4aa7-a402-37cf213b1bf7","Type":"ContainerStarted","Data":"cb4f20d5e81d78b4591560bde103bb751a841bcb83a837f5980665cd673f91cb"} Jul 22 03:07:47.234045 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:47.233635 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" event={"ID":"c687d6f1-4272-4aa7-a402-37cf213b1bf7","Type":"ContainerStarted","Data":"f80f5ccd64a66f43dc8a7caa15eb659248c66cfd038e5baf7ed2fce446a6114b"} Jul 22 03:07:47.234045 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:47.233702 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:47.255612 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:47.255563 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" podStartSLOduration=2.2555483929999998 podStartE2EDuration="2.255548393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:07:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:07:47.254040826 +0000 UTC m=+2898.162905821" watchObservedRunningTime="2026-07-22 03:07:47.255548393 +0000 UTC m=+2898.164413456" Jul 22 03:07:47.987065 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:47.987034 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zgnxf_c3fb8521-f63a-4773-ad73-73ae63e8b4d2/dns/0.log" Jul 22 03:07:48.003074 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:48.003054 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zgnxf_c3fb8521-f63a-4773-ad73-73ae63e8b4d2/kube-rbac-proxy/0.log" Jul 22 03:07:48.080708 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:48.080678 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wkh4j_90aed964-e484-4a30-a581-fc933f03f780/dns-node-resolver/0.log" Jul 22 03:07:48.607045 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:48.607010 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-68dcf7c957-6cgls_7d797fd6-18bb-420d-b40a-89b76860ae09/registry/0.log" Jul 22 03:07:48.645548 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:48.645524 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rzszs_519f47d6-aef0-4eae-8eba-240fe5cadfed/node-ca/0.log" Jul 22 03:07:49.586267 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:49.586235 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nnp5f_218745d1-9832-4760-b46f-3661fa7f4e5a/istio-proxy/0.log" Jul 22 03:07:49.750001 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:49.749977 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c4bb8965-gt6rw_b35918df-d4bf-4130-8e89-9a9b9ca4eb95/kube-auth-proxy/0.log" Jul 22 03:07:49.780524 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:49.780497 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hlntg_42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6/istio-proxy/0.log" Jul 22 03:07:49.808473 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:49.808438 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hlntg_42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6/istio-proxy/1.log" Jul 22 03:07:50.574973 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:50.574947 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mwwl8_1a22a50d-1436-409d-af35-9dc1907dd74e/serve-healthcheck-canary/0.log" Jul 22 03:07:51.118733 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:51.118701 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gmw9c_13baf0c2-a61b-4602-970d-6bde2895bc5e/kube-rbac-proxy/0.log" Jul 22 03:07:51.137859 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:51.137814 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gmw9c_13baf0c2-a61b-4602-970d-6bde2895bc5e/exporter/0.log" Jul 22 03:07:51.157695 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:51.157672 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gmw9c_13baf0c2-a61b-4602-970d-6bde2895bc5e/extractor/0.log" Jul 22 03:07:53.248614 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:53.248588 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-6q9cg" Jul 22 03:07:53.342625 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:53.342602 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-57frf_5cd5e93b-ab6c-464b-a914-02ee4d72f695/manager/0.log" Jul 22 03:07:53.513684 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:53.513602 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-74cxr_32de991c-21b2-4846-94b6-52923946c2d4/manager/0.log" Jul 22 03:07:53.635890 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:53.635852 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-bqxlc_ef9503af-c09f-4cfc-a58d-b459e17ff907/server/0.log" Jul 22 03:07:53.698664 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:53.698632 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-wfh7b_42c5d187-3867-4a46-8f7c-980f572dc26a/manager/0.log" Jul 22 03:07:53.847006 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:53.846931 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb9fd785c-xr8f8_e8e68259-7a67-470c-b249-733ac647e978/manager/0.log" Jul 22 03:07:53.865140 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:53.865116 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb9fd785c-xr8f8_e8e68259-7a67-470c-b249-733ac647e978/copy-manifests/0.log" Jul 22 03:07:55.288981 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:55.288953 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-sm7br_168cf675-4e4b-4f49-9c23-f63150dbcd19/openshift-lws-operator/0.log" Jul 22 03:07:56.146544 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:56.146458 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nnp5f_218745d1-9832-4760-b46f-3661fa7f4e5a/istio-proxy/0.log" Jul 22 03:07:56.295215 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:56.295186 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c4bb8965-gt6rw_b35918df-d4bf-4130-8e89-9a9b9ca4eb95/kube-auth-proxy/0.log" Jul 22 03:07:56.311132 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:56.311101 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hlntg_42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6/istio-proxy/0.log" Jul 22 03:07:56.336888 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:56.336855 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hlntg_42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6/istio-proxy/1.log" Jul 22 03:07:57.616093 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:57.616058 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nnp5f_218745d1-9832-4760-b46f-3661fa7f4e5a/istio-proxy/0.log" Jul 22 03:07:57.759473 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:57.759444 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c4bb8965-gt6rw_b35918df-d4bf-4130-8e89-9a9b9ca4eb95/kube-auth-proxy/0.log" Jul 22 03:07:57.778051 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:57.778017 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hlntg_42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6/istio-proxy/0.log" Jul 22 03:07:57.805932 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:07:57.805900 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hlntg_42dd0d7a-f7b4-4cec-b1f2-ee1a56b3a1d6/istio-proxy/1.log" Jul 22 03:08:00.338941 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:08:00.338911 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-grvw8_25bca1ce-620d-4f9b-8f60-279b6e21668e/migrator/0.log" Jul 22 03:08:00.358479 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:08:00.358452 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-grvw8_25bca1ce-620d-4f9b-8f60-279b6e21668e/graceful-termination/0.log" Jul 22 03:08:01.838884 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:08:01.838856 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7h5ws_ce0f9c32-6e0b-4f17-a4a6-59aea33b8ab1/kube-multus/0.log" Jul 22 03:08:02.296261 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:08:02.296233 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m5st4_a16f14c0-d854-40e4-8151-abebbe9b1176/kube-multus-additional-cni-plugins/0.log" Jul 22 03:08:02.330289 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:08:02.330260 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m5st4_a16f14c0-d854-40e4-8151-abebbe9b1176/egress-router-binary-copy/0.log" Jul 22 03:08:02.359524 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:08:02.359493 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m5st4_a16f14c0-d854-40e4-8151-abebbe9b1176/cni-plugins/0.log" Jul 22 03:08:02.383131 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:08:02.383099 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m5st4_a16f14c0-d854-40e4-8151-abebbe9b1176/bond-cni-plugin/0.log" Jul 22 03:08:02.406998 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:08:02.406973 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m5st4_a16f14c0-d854-40e4-8151-abebbe9b1176/routeoverride-cni/0.log" Jul 22 03:08:02.439837 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:08:02.439790 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m5st4_a16f14c0-d854-40e4-8151-abebbe9b1176/whereabouts-cni-bincopy/0.log" Jul 22 03:08:02.471377 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:08:02.471353 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m5st4_a16f14c0-d854-40e4-8151-abebbe9b1176/whereabouts-cni/0.log" Jul 22 03:08:02.668384 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:08:02.668309 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mxnss_72a0ac8c-4aaf-45c2-bd02-41b8e80e770b/network-metrics-daemon/0.log" Jul 22 03:08:02.690575 ip-10-0-132-119 kubenswrapper[2557]: I0722 03:08:02.690545 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mxnss_72a0ac8c-4aaf-45c2-bd02-41b8e80e770b/kube-rbac-proxy/0.log"